-- Logs begin at Tue 2022-09-13 06:20:49 UTC, end at Tue 2022-09-13 07:19:34 UTC. -- Sep 13 06:46:31.510987 dv-cinder-856941-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 13 06:46:31.519289 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Sep 13 06:46:31 2022] *** Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: nodename: dv-cinder-856941-4-iscsi Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: machine: x86_64 Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: clock source: unix Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: pcre jit disabled Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: detected number of CPU cores: 4 Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: current working directory: / Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: detected binary path: /usr/bin/uwsgi-core Sep 13 06:46:31.521291 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: your processes number limit is 47680 Sep 13 06:46:31.521885 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: your memory page size is 4096 bytes Sep 13 06:46:31.521885 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: detected max file descriptor number: 2048 Sep 13 06:46:31.521885 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: lock engine: pthread robust mutexes Sep 13 06:46:31.521885 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: thunder lock: enabled Sep 13 06:46:31.521885 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 13 06:46:31.521885 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 13 06:46:31.561329 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: Python main interpreter initialized at 0x55fc6e6f39c0 Sep 13 06:46:31.561329 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: python threads support enabled Sep 13 06:46:31.561329 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: your server socket listen backlog is limited to 100 connections Sep 13 06:46:31.561329 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: your mercy for graceful operations on workers is 90 seconds Sep 13 06:46:31.561688 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: mapped 403077 bytes (393 KB) for 2 cores Sep 13 06:46:31.561768 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: *** Operational MODE: preforking *** Sep 13 06:46:31.562021 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: *** uWSGI is running in multiple interpreter mode *** Sep 13 06:46:31.562021 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: spawned uWSGI master process (pid: 81488) Sep 13 06:46:31.562255 dv-cinder-856941-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 13 06:46:31.562760 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: spawned uWSGI worker 1 (pid: 81499, cores: 1) Sep 13 06:46:31.563098 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: spawned uWSGI worker 2 (pid: 81500, cores: 1) Sep 13 06:46:31.566141 dv-cinder-856941-4-iscsi devstack@keystone.service[81488]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 13 06:46:35.798007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 06:46:35.798007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: from cryptography.utils import int_from_bytes Sep 13 06:46:35.798881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 06:46:35.798881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: from cryptography.utils import int_from_bytes Sep 13 06:46:35.799320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 06:46:35.799320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: from cryptography.utils import int_from_bytes Sep 13 06:46:35.799421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 13 06:46:35.799421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: from cryptography.utils import int_from_bytes Sep 13 06:46:35.848139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 06:46:35.848351 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 13 06:46:35.848437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 06:46:35.848513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 13 06:46:35.848578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] command line args: [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 06:46:35.848638 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] command line args: [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 13 06:46:35.848704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 06:46:35.848841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 13 06:46:35.848967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ================================================================================ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 06:46:35.849048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ================================================================================ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 13 06:46:35.849115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] admin_token = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.849319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] admin_token = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.849454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config_dir = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.849531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config_dir = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.849598 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.849663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.849727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] config_source = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.849796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] config_source = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.849856 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] debug = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.850017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] debug = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.850134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.850346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.850444 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.850512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.850571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.850627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.850840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.850968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_config_append = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_config_append = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_dir = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.851979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_dir = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.852083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.852186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.852287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.852385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.852471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.852594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.852676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.852748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.852880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.853914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.854025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.854148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.854267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.854399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.854487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.854602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.854757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.854927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.855020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.855095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.855231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.855323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.855726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.855861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.25/identity {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.855953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] publish_errors = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] publish_errors = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.856990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.857156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] standard_threads = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.857897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] standard_threads = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_journal = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_json = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_journal = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_stderr = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_json = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.858988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] use_syslog = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.859070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_stderr = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.859070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] use_syslog = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.859205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.859275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 13 06:46:35.859346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.859455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.859548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.859621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.859698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.859772 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.859849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.859921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.860002 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.860077 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.860146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.860336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.860460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.860540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.860626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.860797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.860922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.861003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.861075 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.861274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.861410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.861498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.861572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.861735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.861858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.861940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.862027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.862232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.862379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.862468 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.862543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.862718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.862900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.862982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.863051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.863166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.863246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.863439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.863654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.863654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.863796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.863875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.863946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.864148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.864285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.864366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.864439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.864625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.864748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.864840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.864922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.865077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.865200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.865279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.865351 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.865511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.865634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.865723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.865797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.865991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.866152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.866247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.866323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.866445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.866528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.866683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.866763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.866925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.867007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.867162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.867251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.867401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.867483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.867625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.867706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.867854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.867936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.868082 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.868215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.868300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.868381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.868527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.868608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.868751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.868833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.868983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.869059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.869195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.869282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.869430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.869512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.869656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.869746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.869890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.869972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.870181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.870267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.870349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.870421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.870577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.870660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.870816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.870927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.871032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.871128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.871252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.871359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.871482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.871595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.871703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.871799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.871940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.872063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.872144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.872223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.872388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.872478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.872613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.872695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.872931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.873085 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.873263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.873662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.873755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.873830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.873962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874520 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.874835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.875021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.875102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.875260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.875353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.875514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.875616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.875733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.875808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.875977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.876070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.876208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.876291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.876441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.876525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.876672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.876762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.876907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.876990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.877136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.877229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.877377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.877467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.877617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.877698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.877819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.877895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.878061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.878181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.878319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.878457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.878534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.878650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.878781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.878875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.879010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.879092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.879251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.879338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.879498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.879595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.879719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.879808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.879956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.880034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.880192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.880347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.880422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.880492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.880644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.880721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.880876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.880962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.881104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.881183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.881332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.881409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.881559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.881635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.881788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.881865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.882013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.882164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.882271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.882349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.882504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.882583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.882740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.882817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.882969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.883046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.883217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.883295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.883470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.883546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.883707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.883784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.883940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.external = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.884028 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.884167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.884255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.external = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.884397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.884474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.884675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.884753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.884901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.password = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.884980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.885128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] auth.token = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.885213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.password = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.885358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] auth.token = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.885441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.885586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.885670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.885821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.885899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.886045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.886142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.886312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.886393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.886553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.886631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.886780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.886858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.887011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.887094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.887245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.887326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.887485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.887564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.887727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.887808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.887964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.888044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.888202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.888280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.888433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.888577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.888809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889354 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.889879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.890013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.890155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.890296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.890421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.890534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.890665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.890785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.890924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.891003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.891161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.891239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.891399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.891479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.891647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.891727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.891894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.891973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.892126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.892205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.892362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.892440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.892597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.892682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.892828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.892906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.893072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.893149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.893306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.893391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.893550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.893629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.893785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.893863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.894020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.894235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.894316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.894388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.894505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.894584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.894735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.894821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.894977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.895055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.895204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.895281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.895436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.895522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.895673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.895751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.895917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.896002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.896145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.896230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.896371 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.896449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.896597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.896675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.896829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.896914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.897068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.897145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.897309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.897387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.897553 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.897639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.897793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.897872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.898041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.898138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.898307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.898443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.898543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.898622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.898769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.898848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.899992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.900152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.900228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.900376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.900462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.900617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.900694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.900858 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.900935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.901088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.901174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.901321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.901398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.901559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.901636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.901785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.901873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.902019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.902136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.902266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.902345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.902497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.902565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.902727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.902804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.902959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.903036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.903190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.903267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.903425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.903533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.903775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.903853 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.904026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.904179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.904650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.904650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.904787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.904787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.904926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.905001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.905073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.905249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.905335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.905483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.905570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.905717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.905795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.905949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.906028 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.906209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.906302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.906445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.906523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.906681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.906758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.906926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.907012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.907161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.907239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.907402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.907479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.907633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.907719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.907861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.907939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.908094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.908172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.908325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.908411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.908556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.908634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.908783 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.908860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.909020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.909110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.909252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.909331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.909490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.909567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.909731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.909818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.909973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.910052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.910224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.910305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.910468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.910559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.910701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.910778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.910930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.911007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.911162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.911249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.911394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.911473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.911631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.911708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.911866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.911955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.912096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.912174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.912346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.912425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.912572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.912662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.912812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.912895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.913044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.913121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.913274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.913360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.913507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.913594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.913741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.913819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.913977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.914063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.914227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.914306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.914461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.914538 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.914701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.914798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.914936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.915015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.915178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.915261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.915412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.915499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.915658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.915736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.915891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.915968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.916124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.916209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.916367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.916445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.916602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.916679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.916834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.916986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.917062 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.917134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.917300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.917384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.917526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.917605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.917769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.917846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.918001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.918133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.918265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.918345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.918509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.918587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.918751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.918830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.918991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.919071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.919234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.919312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.919476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.919554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.919740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.919827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.920288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.920366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.920439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.920512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.920594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.920669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.920742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.920838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.921035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.921113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.921271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.921351 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.921519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.921700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.921779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.921943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.922030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.922224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.922305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.922477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.922555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.922728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.922820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.922977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.923056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.923214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.923291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.923442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.923520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.923676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.923754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.923905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.923983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.924152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.924231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.924384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.924461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.924623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.924700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.924856 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.924935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.925088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.925167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.925318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.925396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.925552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.925631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.925787 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.925866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.926017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.926113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.926281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.926361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.926535 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.926613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.926779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.926856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.927013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.927093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.927255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.927333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.927496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.927574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.927732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.927811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.927964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.928041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.928199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.928277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.928430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.928510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.928671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.928748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.928911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.928989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.929146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.929225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.929381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.929469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.929641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.929719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.929870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.929959 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.930150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.930248 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.930382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.930460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.930615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.930693 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.930852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.930938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.931084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.931163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.931316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.931395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.931545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.931632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.931776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.931854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.932009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.932087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.932239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.932325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.932473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.932551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.932702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.932779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.932943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.933368 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.933517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.933595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.933757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.933834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.933983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.934073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.934242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.934321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.934475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.934555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.934627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81500) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 06:46:35.934712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.934944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.935181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 13 06:46:35.935393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81499) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 13 06:46:35.963822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:35.964018 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:35.964090 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:35.964248 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:35.964376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:35.964540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:35.972616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:35.972940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.000527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.000736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.000868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.001093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.053059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.053404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.100872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.101089 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.102190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.102428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.258877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259444 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.259937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.260017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.282916 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.283113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.283239 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.283333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.283414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.283550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.283670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.283753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.283830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.283948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.284029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.284159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.284238 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.284410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.296334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.299064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.309261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.309499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.309718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.310034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.310241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.310342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.310619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81499) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.311000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.311203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.311459 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.311691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.311881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.312127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.312310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81500) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 13 06:46:36.327441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.327658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 06:46:36.327786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.327942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 06:46:36.328067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.328160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 06:46:36.328240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.328427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.328523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 13 06:46:36.328690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.328794 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.328921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.329041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.329165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.329325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.329447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.338788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.339013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.339146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.339263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.339356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.339534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.339658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.339838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.346765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.346894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.349674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.349806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.349959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.350111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.350241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.350378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.350555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.350648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.350773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.350855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.351025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.351127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.351321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.351454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.351582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.351722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.351861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.351993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.352121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.352251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.352377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.352504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.374910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.375104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.376364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.376570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.376671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.376853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.381255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.381490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.381622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.381785 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.381929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.382034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.388074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.388362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.388577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.388793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.394112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.394420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.394697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.394825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.394976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.395165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.395257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.395380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.395503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.395626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.395810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.395936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.396064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.396192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.396319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.396597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.411387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.411665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.412115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.412393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.415492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.416280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.418750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.419043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.419523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.419860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.423530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.424087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.426677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.427227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.429881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.430196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.430424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.430556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.430705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.430792 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.431015 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.431144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.431343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.431474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.431632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.431887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.447719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.447848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.448002 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.448131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.448256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.448405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.448490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.448612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.455371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.455560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.456893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.457093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.458420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.458649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.458775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.458903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.461364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.461604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.461730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.461859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.461950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.462190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.462317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.462443 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.462595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.462866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.473827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.473949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.474112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.474202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.474418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.474563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.474748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.474925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.475081 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.475175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.475359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.475485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.475674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.475799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.490973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.491119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.493850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.493977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.496728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.496856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.498240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.498404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.499697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.499860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.499955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.500137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.500361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.500564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.506710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.506950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.508179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.508404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.511130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.511251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.511465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.511630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.511725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.511949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.512164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.512249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.519843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.520073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.520189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.520389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.520516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.520720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.527253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81500) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.527535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.527661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81499) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 13 06:46:36.527801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.527885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.528049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.528159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.528321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.528412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.528622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.528751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.528889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.528984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.529193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.529336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.529448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.529750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.529877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.529999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.530118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.530326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.530474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.530561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.530638 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.530884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.531012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.531186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81499) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.531328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81500) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 13 06:46:36.565220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fc6e6f39c0 pid: 81500 (default app) Sep 13 06:46:36.565482 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55fc6e6f39c0 pid: 81499 (default app) Sep 13 06:46:36.568604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7f865-5ba6-401f-9430-40ad8f9ed6db None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:36.568890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7f865-5ba6-401f-9430-40ad8f9ed6db None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:36.569159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7f865-5ba6-401f-9430-40ad8f9ed6db None None] PATH_INFO: `/v3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:36.573346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1/1] 10.222.0.25 () {50 vars in 791 bytes} [Tue Sep 13 06:46:36 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:43.391883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56d3b7-524d-4569-8c33-4809b0f36426 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:43.392566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56d3b7-524d-4569-8c33-4809b0f36426 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:43.392566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56d3b7-524d-4569-8c33-4809b0f36426 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:43.396777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1/2] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:46:43.402306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:43.402816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:43.402816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:43.472679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: INFO dbcounter [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] Registered counter for database keystone Sep 13 06:46:43.493868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG oslo_db.sqlalchemy.engines [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81500) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 06:46:43.505832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writer thread running {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 06:46:43.693284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG passlib.handlers.bcrypt [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81500) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 06:46:43.693555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG passlib.handlers.bcrypt [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81500) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 06:46:43.752238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:43.753856 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:43.895923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 2/3] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:43 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 06:46:44.386646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56d3b7-524d-4569-8c33-4809b0f36426 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:44.466403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: INFO dbcounter [None req-7a56d3b7-524d-4569-8c33-4809b0f36426 None None] Registered counter for database keystone Sep 13 06:46:44.491797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG oslo_db.sqlalchemy.engines [None req-7a56d3b7-524d-4569-8c33-4809b0f36426 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81499) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 13 06:46:44.502637 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writer thread running {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 13 06:46:44.621754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a56d3b7-524d-4569-8c33-4809b0f36426 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:44.627236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ddd7c2-47da-4ef6-803a-026ad4da46c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:44.627867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddd7c2-47da-4ef6-803a-026ad4da46c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:44.628148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddd7c2-47da-4ef6-803a-026ad4da46c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:44.628417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ddd7c2-47da-4ef6-803a-026ad4da46c7 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:44.637536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddd7c2-47da-4ef6-803a-026ad4da46c7 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:44.688084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WARNING keystone.common.rbac_enforcer.enforcer [None req-71ddd7c2-47da-4ef6-803a-026ad4da46c7 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 06:46:44.688423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ddd7c2-47da-4ef6-803a-026ad4da46c7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:44.698743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 2/4] 10.222.0.25 () {58 vars in 1324 bytes} [Tue Sep 13 06:46:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 06:46:44.708315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:44.730183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee6329b-1dc9-4a98-a8d2-f91455e49f77 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:44.735799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae6cb7-6173-4eaa-a2fd-e18d04834cba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:44.736946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae6cb7-6173-4eaa-a2fd-e18d04834cba admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:44.737232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae6cb7-6173-4eaa-a2fd-e18d04834cba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:44.737516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae6cb7-6173-4eaa-a2fd-e18d04834cba admin admin] PATH_INFO: `/v3/projects/2becfff2f1104b709c0a7babcac266dd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:44.739172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae6cb7-6173-4eaa-a2fd-e18d04834cba admin admin] RBAC: Authorizing `identity:get_project(project_id=2becfff2f1104b709c0a7babcac266dd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:44.791362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WARNING keystone.common.rbac_enforcer.enforcer [None req-35ae6cb7-6173-4eaa-a2fd-e18d04834cba admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 13 06:46:44.791699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ae6cb7-6173-4eaa-a2fd-e18d04834cba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:44.796336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 3/5] 10.222.0.25 () {56 vars in 1207 bytes} [Tue Sep 13 06:46:44 2022] GET /identity/v3/projects/2becfff2f1104b709c0a7babcac266dd => generated 344 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:46.911705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c77da-a2c3-454b-b19e-15551f875bbc None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:46.912258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c77da-a2c3-454b-b19e-15551f875bbc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:46.912258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6c77da-a2c3-454b-b19e-15551f875bbc None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:46.917567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 3/6] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:46 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:46:46.924663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efcceb-cbca-4ac5-b30a-520c720ecbbf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:46.925405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efcceb-cbca-4ac5-b30a-520c720ecbbf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:46.925553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4efcceb-cbca-4ac5-b30a-520c720ecbbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:46.950852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d4efcceb-cbca-4ac5-b30a-520c720ecbbf None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:46.952564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d4efcceb-cbca-4ac5-b30a-520c720ecbbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:46.961701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 4/7] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:46 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 13 06:46:47.226274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c77da-a2c3-454b-b19e-15551f875bbc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:47.240995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6c77da-a2c3-454b-b19e-15551f875bbc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:47.246486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c043bd9-4fff-4328-9168-a98e30f297ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:47.247094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c043bd9-4fff-4328-9168-a98e30f297ac admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:47.247375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c043bd9-4fff-4328-9168-a98e30f297ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:47.247646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c043bd9-4fff-4328-9168-a98e30f297ac admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:47.256505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c043bd9-4fff-4328-9168-a98e30f297ac admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:47.258320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c043bd9-4fff-4328-9168-a98e30f297ac admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:47.267706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 4/8] 10.222.0.25 () {58 vars in 1324 bytes} [Tue Sep 13 06:46:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 06:46:47.274025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efcceb-cbca-4ac5-b30a-520c720ecbbf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:47.289522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4efcceb-cbca-4ac5-b30a-520c720ecbbf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:47.298184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b32a9-00d8-4a2a-ad1a-55d9a11b1081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:47.298828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b32a9-00d8-4a2a-ad1a-55d9a11b1081 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:47.299106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b32a9-00d8-4a2a-ad1a-55d9a11b1081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:47.299374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b32a9-00d8-4a2a-ad1a-55d9a11b1081 admin admin] PATH_INFO: `/v3/users/d3bdb62de62349b6b321bb705e25f4be` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:47.300816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b32a9-00d8-4a2a-ad1a-55d9a11b1081 admin admin] RBAC: Authorizing `identity:get_user(user_id=d3bdb62de62349b6b321bb705e25f4be)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:47.302700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b32a9-00d8-4a2a-ad1a-55d9a11b1081 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:47.304224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 5/9] 10.222.0.25 () {56 vars in 1198 bytes} [Tue Sep 13 06:46:47 2022] GET /identity/v3/users/d3bdb62de62349b6b321bb705e25f4be => generated 247 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:49.543941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c13e34-e213-4ab1-a0ec-6514b3bd27d2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:49.544528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c13e34-e213-4ab1-a0ec-6514b3bd27d2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:49.544528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c13e34-e213-4ab1-a0ec-6514b3bd27d2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:49.545981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 5/10] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:46:49.552075 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda526d4-e8f0-4b87-b2dd-b48cad25a8df None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:49.552230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44132ba-479b-422f-817e-6b5bd93bc727 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:49.552645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda526d4-e8f0-4b87-b2dd-b48cad25a8df None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:49.552734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44132ba-479b-422f-817e-6b5bd93bc727 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:49.552830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda526d4-e8f0-4b87-b2dd-b48cad25a8df None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:49.552909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44132ba-479b-422f-817e-6b5bd93bc727 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:49.553786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 6/11] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:49 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:46:49.559436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1191ec1-790d-496f-abe6-f36a9ba7e7ed None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:49.559737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1191ec1-790d-496f-abe6-f36a9ba7e7ed None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:49.559985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1191ec1-790d-496f-abe6-f36a9ba7e7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:49.571411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b44132ba-479b-422f-817e-6b5bd93bc727 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:49.572567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b44132ba-479b-422f-817e-6b5bd93bc727 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:49.602293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG passlib.handlers.bcrypt [None req-b1191ec1-790d-496f-abe6-f36a9ba7e7ed None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81499) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 13 06:46:49.602672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG passlib.handlers.bcrypt [None req-b1191ec1-790d-496f-abe6-f36a9ba7e7ed None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81499) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 13 06:46:49.651054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b1191ec1-790d-496f-abe6-f36a9ba7e7ed None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:49.652659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b1191ec1-790d-496f-abe6-f36a9ba7e7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:49.655926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 6/12] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:46:49.754847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 7/13] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:46:49.926016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44132ba-479b-422f-817e-6b5bd93bc727 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:49.952430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:50.014326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1191ec1-790d-496f-abe6-f36a9ba7e7ed None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:50.036450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44132ba-479b-422f-817e-6b5bd93bc727 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:50.071995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee4c4a-0dd3-466f-876a-8830a450b014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:50.072620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee4c4a-0dd3-466f-876a-8830a450b014 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:50.072902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee4c4a-0dd3-466f-876a-8830a450b014 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:50.073186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbee4c4a-0dd3-466f-876a-8830a450b014 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:50.117937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1191ec1-790d-496f-abe6-f36a9ba7e7ed None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:50.120452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:50.141570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee4c4a-0dd3-466f-876a-8830a450b014 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:50.143083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbee4c4a-0dd3-466f-876a-8830a450b014 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:50.157331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb350-98cb-473a-8a33-50ac43228a42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:50.158597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb350-98cb-473a-8a33-50ac43228a42 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:50.158881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb350-98cb-473a-8a33-50ac43228a42 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:50.159150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bfb350-98cb-473a-8a33-50ac43228a42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:50.166170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:50.171570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bfb350-98cb-473a-8a33-50ac43228a42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e7bfb350-98cb-473a-8a33-50ac43228a42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:46:50.180214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:46:50.181301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:46:50.182200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 8/14] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:46:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:46:50.187492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb350-98cb-473a-8a33-50ac43228a42 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:50.243727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 7/15] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:46:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:46:50.249441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee4c4a-0dd3-466f-876a-8830a450b014 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:50.288561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bfb350-98cb-473a-8a33-50ac43228a42 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:50.321365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89ec8a-2f70-42a0-b887-6f1abbb3c117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:50.321976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89ec8a-2f70-42a0-b887-6f1abbb3c117 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:50.322280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89ec8a-2f70-42a0-b887-6f1abbb3c117 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:50.322549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b89ec8a-2f70-42a0-b887-6f1abbb3c117 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:50.323220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89ec8a-2f70-42a0-b887-6f1abbb3c117 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:46:50.323521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89ec8a-2f70-42a0-b887-6f1abbb3c117 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:50.324969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b89ec8a-2f70-42a0-b887-6f1abbb3c117 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:50.336962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 9/16] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:46:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:46:50.342745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89ec8a-2f70-42a0-b887-6f1abbb3c117 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:50.346450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbee4c4a-0dd3-466f-876a-8830a450b014 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:50.369723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:50.370563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:50.382802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864e126-f39c-4cea-8ac4-ddd3b0e5c50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:50.383438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864e126-f39c-4cea-8ac4-ddd3b0e5c50b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:50.383715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864e126-f39c-4cea-8ac4-ddd3b0e5c50b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:50.383969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864e126-f39c-4cea-8ac4-ddd3b0e5c50b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:50.394187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864e126-f39c-4cea-8ac4-ddd3b0e5c50b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:50.395683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864e126-f39c-4cea-8ac4-ddd3b0e5c50b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5864e126-f39c-4cea-8ac4-ddd3b0e5c50b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:46:50.418029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:46:50.419004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:46:50.419865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:46:50.419865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:46:50.419865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:46:50.419865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 8/17] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:46:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:46:50.425347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864e126-f39c-4cea-8ac4-ddd3b0e5c50b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:50.448104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b89ec8a-2f70-42a0-b887-6f1abbb3c117 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:50.484122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef73567-1a0f-4c10-905e-401e76cd9687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:50.484762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef73567-1a0f-4c10-905e-401e76cd9687 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:50.485037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef73567-1a0f-4c10-905e-401e76cd9687 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:50.485313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef73567-1a0f-4c10-905e-401e76cd9687 None admin] PATH_INFO: `/v3/users/d3bdb62de62349b6b321bb705e25f4be` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:50.486947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef73567-1a0f-4c10-905e-401e76cd9687 None admin] RBAC: Authorizing `identity:get_user(user_id=d3bdb62de62349b6b321bb705e25f4be)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:50.488435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef73567-1a0f-4c10-905e-401e76cd9687 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:50.490014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 10/18] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:46:50 2022] GET /identity/v3/users/d3bdb62de62349b6b321bb705e25f4be => generated 247 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:46:50.495442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef73567-1a0f-4c10-905e-401e76cd9687 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:50.528243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864e126-f39c-4cea-8ac4-ddd3b0e5c50b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:50.562765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1dafd-3712-41d4-a19d-762f6ab86a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:50.563395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1dafd-3712-41d4-a19d-762f6ab86a3c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:50.563658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1dafd-3712-41d4-a19d-762f6ab86a3c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:50.563928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e1dafd-3712-41d4-a19d-762f6ab86a3c None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:50.564578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1dafd-3712-41d4-a19d-762f6ab86a3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:46:50.564881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1dafd-3712-41d4-a19d-762f6ab86a3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:50.566305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e1dafd-3712-41d4-a19d-762f6ab86a3c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:50.584522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 9/19] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:46:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:46:50.591823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:50.600148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef73567-1a0f-4c10-905e-401e76cd9687 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:50.636705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9987a8-5c19-4461-8163-a1db48c5041f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:50.637370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9987a8-5c19-4461-8163-a1db48c5041f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:50.637692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9987a8-5c19-4461-8163-a1db48c5041f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:50.637960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9987a8-5c19-4461-8163-a1db48c5041f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:50.639287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9987a8-5c19-4461-8163-a1db48c5041f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:50.640800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9987a8-5c19-4461-8163-a1db48c5041f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:50.642497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 11/20] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:46:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:50.648846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1dafd-3712-41d4-a19d-762f6ab86a3c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:50.688940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:50.767112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e1dafd-3712-41d4-a19d-762f6ab86a3c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:50.801758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff508a8-e39f-4bb5-a07a-b066c32eb82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:50.802416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff508a8-e39f-4bb5-a07a-b066c32eb82f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:50.802692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff508a8-e39f-4bb5-a07a-b066c32eb82f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:50.802953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff508a8-e39f-4bb5-a07a-b066c32eb82f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:50.803687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff508a8-e39f-4bb5-a07a-b066c32eb82f None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.domain.id=default, user.id=d3bdb62de62349b6b321bb705e25f4be) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:46:50.804013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff508a8-e39f-4bb5-a07a-b066c32eb82f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:50.805481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff508a8-e39f-4bb5-a07a-b066c32eb82f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:50.812736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 10/21] 10.222.0.25 () {56 vars in 1322 bytes} [Tue Sep 13 06:46:50 2022] GET /identity/v3/role_assignments?user.id=d3bdb62de62349b6b321bb705e25f4be&scope.domain.id=default&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 233 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:53.065627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89b2b3-aea0-46ab-a337-5b827d7e8d64 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:53.066648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89b2b3-aea0-46ab-a337-5b827d7e8d64 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:53.067146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae89b2b3-aea0-46ab-a337-5b827d7e8d64 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:53.068967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 12/22] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:46:53.074940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd035b-b0aa-4ce3-8d8e-5ef2e3d2631b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:53.075777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd035b-b0aa-4ce3-8d8e-5ef2e3d2631b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:53.076339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd035b-b0aa-4ce3-8d8e-5ef2e3d2631b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:53.098741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-73fd035b-b0aa-4ce3-8d8e-5ef2e3d2631b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:53.100278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-73fd035b-b0aa-4ce3-8d8e-5ef2e3d2631b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:53.200352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 11/23] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:46:53.332406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28838d8-8ee1-4291-bf6b-2e234e5bdea6 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:53.333045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28838d8-8ee1-4291-bf6b-2e234e5bdea6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:53.333528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28838d8-8ee1-4291-bf6b-2e234e5bdea6 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:53.335389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 13/24] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:46:53.340357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89151f3-b23a-41ea-add9-7c66533d435b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:53.340644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89151f3-b23a-41ea-add9-7c66533d435b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:53.340923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89151f3-b23a-41ea-add9-7c66533d435b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:53.362361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b89151f3-b23a-41ea-add9-7c66533d435b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:53.363906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b89151f3-b23a-41ea-add9-7c66533d435b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:53.396961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:53.458271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 12/25] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:46:53.578737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dee85f-e0be-4d27-8a8b-d3b73026e11d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:53.579106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dee85f-e0be-4d27-8a8b-d3b73026e11d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:53.579303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dee85f-e0be-4d27-8a8b-d3b73026e11d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:53.580656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 14/26] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:46:53.587436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3e51-60ca-43eb-af24-c09024ac32fe None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:53.587666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3e51-60ca-43eb-af24-c09024ac32fe None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:53.587946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcd3e51-60ca-43eb-af24-c09024ac32fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:53.610610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3fcd3e51-60ca-43eb-af24-c09024ac32fe None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:53.612007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3fcd3e51-60ca-43eb-af24-c09024ac32fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:53.701323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 13/27] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:46:53.709676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dee85f-e0be-4d27-8a8b-d3b73026e11d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:53.720958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c0a1e-4338-44a9-a32c-643ef7a4446b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:53.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c0a1e-4338-44a9-a32c-643ef7a4446b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:53.721526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c0a1e-4338-44a9-a32c-643ef7a4446b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:53.722907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 14/28] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:46:53.731416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47ce4e-25b8-49d5-8f92-84293409325b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:53.731698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47ce4e-25b8-49d5-8f92-84293409325b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:53.732032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb47ce4e-25b8-49d5-8f92-84293409325b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:53.753712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fb47ce4e-25b8-49d5-8f92-84293409325b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:53.755002 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fb47ce4e-25b8-49d5-8f92-84293409325b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:53.872020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 15/29] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 06:46:53.891739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dee85f-e0be-4d27-8a8b-d3b73026e11d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:53.922756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47ce4e-25b8-49d5-8f92-84293409325b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:53.943332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5682aa-24e8-4202-9106-75ad1cbef57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:53.943986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5682aa-24e8-4202-9106-75ad1cbef57e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:53.944269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5682aa-24e8-4202-9106-75ad1cbef57e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:53.944543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5682aa-24e8-4202-9106-75ad1cbef57e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:53.953975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:53.977316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5682aa-24e8-4202-9106-75ad1cbef57e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:53.978849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb5682aa-24e8-4202-9106-75ad1cbef57e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bb5682aa-24e8-4202-9106-75ad1cbef57e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:46:54.006026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:46:54.006969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:46:54.007774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:46:54.007774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:46:54.007774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:46:54.010215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 15/30] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:46:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:46:54.017175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5682aa-24e8-4202-9106-75ad1cbef57e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:54.038289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:54.042009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb47ce4e-25b8-49d5-8f92-84293409325b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:54.081824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2dae7-1259-4ff7-b0bf-8796d9f7c2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:54.083282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2dae7-1259-4ff7-b0bf-8796d9f7c2fd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:54.083869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2dae7-1259-4ff7-b0bf-8796d9f7c2fd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:54.084329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2dae7-1259-4ff7-b0bf-8796d9f7c2fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:54.096904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2dae7-1259-4ff7-b0bf-8796d9f7c2fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-53d2dae7-1259-4ff7-b0bf-8796d9f7c2fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:46:54.104090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:46:54.104952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:46:54.105704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:46:54.107888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 16/31] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:46:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:46:54.119331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2dae7-1259-4ff7-b0bf-8796d9f7c2fd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:54.188246 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5682aa-24e8-4202-9106-75ad1cbef57e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:54.220942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:54.235539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943319a-ce4e-4a11-8759-6185b497b4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:54.236178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943319a-ce4e-4a11-8759-6185b497b4f5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:54.236457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943319a-ce4e-4a11-8759-6185b497b4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:54.236728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3943319a-ce4e-4a11-8759-6185b497b4f5 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:54.237377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943319a-ce4e-4a11-8759-6185b497b4f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:46:54.237676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943319a-ce4e-4a11-8759-6185b497b4f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:54.240865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3943319a-ce4e-4a11-8759-6185b497b4f5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:54.264247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2dae7-1259-4ff7-b0bf-8796d9f7c2fd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:54.275285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 16/32] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:46:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:54.276422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943319a-ce4e-4a11-8759-6185b497b4f5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:54.318761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558d968-2991-4458-a3e0-092ca76b81de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:54.319343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558d968-2991-4458-a3e0-092ca76b81de None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:54.319621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558d968-2991-4458-a3e0-092ca76b81de None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:54.319900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2558d968-2991-4458-a3e0-092ca76b81de None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:54.320645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558d968-2991-4458-a3e0-092ca76b81de None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:46:54.320879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558d968-2991-4458-a3e0-092ca76b81de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:54.322316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2558d968-2991-4458-a3e0-092ca76b81de None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:54.334798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 17/33] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:46:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:46:54.335606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558d968-2991-4458-a3e0-092ca76b81de None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:54.383753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:54.418928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3943319a-ce4e-4a11-8759-6185b497b4f5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:54.448871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2558d968-2991-4458-a3e0-092ca76b81de None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:54.467920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40161ec-ab44-4b23-9b5f-eb512ad0fcad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:54.468541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40161ec-ab44-4b23-9b5f-eb512ad0fcad None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:54.468830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40161ec-ab44-4b23-9b5f-eb512ad0fcad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:54.469105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40161ec-ab44-4b23-9b5f-eb512ad0fcad None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:54.469791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40161ec-ab44-4b23-9b5f-eb512ad0fcad None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:54.471395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40161ec-ab44-4b23-9b5f-eb512ad0fcad None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:54.499727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb4027-81ec-4f85-9cfb-5ae537cd6dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:54.500805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb4027-81ec-4f85-9cfb-5ae537cd6dea None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:54.501272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb4027-81ec-4f85-9cfb-5ae537cd6dea None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:54.501745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb4027-81ec-4f85-9cfb-5ae537cd6dea None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:54.502614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb4027-81ec-4f85-9cfb-5ae537cd6dea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:54.504368 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb4027-81ec-4f85-9cfb-5ae537cd6dea None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:54.517772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:54.605878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 17/34] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Sep 13 06:46:54 2022] POST /identity/v3/roles => generated 225 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:46:54.606910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40161ec-ab44-4b23-9b5f-eb512ad0fcad None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:54.657992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 18/35] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:46:54 2022] POST /identity/v3/projects => generated 301 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:46:54.659073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb4027-81ec-4f85-9cfb-5ae537cd6dea None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:54.882993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40161ec-ab44-4b23-9b5f-eb512ad0fcad None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:54.941392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f475f07-9671-46b3-98d7-73b087a9ffbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:54.942094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475f07-9671-46b3-98d7-73b087a9ffbb None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:54.942436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475f07-9671-46b3-98d7-73b087a9ffbb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:54.942715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f475f07-9671-46b3-98d7-73b087a9ffbb None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:54.943415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f475f07-9671-46b3-98d7-73b087a9ffbb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:54.944978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f475f07-9671-46b3-98d7-73b087a9ffbb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:54.949538 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:54.962994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb4027-81ec-4f85-9cfb-5ae537cd6dea None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:54.967152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 18/36] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Sep 13 06:46:54 2022] POST /identity/v3/roles => generated 219 bytes in 361 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:46:55.002663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58964e2-45db-4bf2-98c3-e98aa5c0248f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:55.002990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58964e2-45db-4bf2-98c3-e98aa5c0248f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:55.003223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58964e2-45db-4bf2-98c3-e98aa5c0248f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:55.004906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 19/37] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:46:55.012353 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8454b09-684b-4649-8c08-a71b2c1946cf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:55.012743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8454b09-684b-4649-8c08-a71b2c1946cf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:55.012905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8454b09-684b-4649-8c08-a71b2c1946cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:55.020545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5453a-02e3-4f55-801f-b9cfbc9a6c4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:55.021607 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5453a-02e3-4f55-801f-b9cfbc9a6c4a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:55.022616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5453a-02e3-4f55-801f-b9cfbc9a6c4a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:55.023120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde5453a-02e3-4f55-801f-b9cfbc9a6c4a None admin] PATH_INFO: `/v3/users/d3bdb62de62349b6b321bb705e25f4be` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:55.025882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5453a-02e3-4f55-801f-b9cfbc9a6c4a None admin] RBAC: Authorizing `identity:get_user(user_id=d3bdb62de62349b6b321bb705e25f4be)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:55.027846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde5453a-02e3-4f55-801f-b9cfbc9a6c4a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:55.030620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 19/38] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:46:54 2022] GET /identity/v3/users/d3bdb62de62349b6b321bb705e25f4be => generated 247 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:55.037383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5453a-02e3-4f55-801f-b9cfbc9a6c4a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:55.038684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b8454b09-684b-4649-8c08-a71b2c1946cf None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:55.040661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b8454b09-684b-4649-8c08-a71b2c1946cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:55.129165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:55.143428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 20/39] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:46:55.185411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde5453a-02e3-4f55-801f-b9cfbc9a6c4a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:55.221352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb755401-c723-44e1-93df-d127af2d33ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:55.222008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb755401-c723-44e1-93df-d127af2d33ff None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:55.222322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb755401-c723-44e1-93df-d127af2d33ff None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:55.222600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb755401-c723-44e1-93df-d127af2d33ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:55.223844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb755401-c723-44e1-93df-d127af2d33ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:55.225325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb755401-c723-44e1-93df-d127af2d33ff None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:55.227149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 20/40] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:46:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:46:55.233215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8454b09-684b-4649-8c08-a71b2c1946cf None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:55.347184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8454b09-684b-4649-8c08-a71b2c1946cf None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:55.361854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506641c-debc-4253-a2e7-906cb65d7fd6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:55.362175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506641c-debc-4253-a2e7-906cb65d7fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:55.362462 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c506641c-debc-4253-a2e7-906cb65d7fd6 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:55.363979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 21/41] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:46:55.368668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a5828-51ad-4db7-9d32-1aa0c039515a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:55.368961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a5828-51ad-4db7-9d32-1aa0c039515a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:55.369218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419a5828-51ad-4db7-9d32-1aa0c039515a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:55.373102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:55.386427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098ef08-8522-4e8f-a0f4-703708080f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:55.387111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098ef08-8522-4e8f-a0f4-703708080f39 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:55.387395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098ef08-8522-4e8f-a0f4-703708080f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:55.387677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8098ef08-8522-4e8f-a0f4-703708080f39 None admin] PATH_INFO: `/v3/domains/default/users/d3bdb62de62349b6b321bb705e25f4be/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:55.388233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-419a5828-51ad-4db7-9d32-1aa0c039515a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:55.389798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098ef08-8522-4e8f-a0f4-703708080f39 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d3bdb62de62349b6b321bb705e25f4be, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:55.389905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-419a5828-51ad-4db7-9d32-1aa0c039515a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:55.391301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8098ef08-8522-4e8f-a0f4-703708080f39 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:55.411976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 21/42] 10.222.0.25 () {58 vars in 1361 bytes} [Tue Sep 13 06:46:55 2022] PUT /identity/v3/domains/default/users/d3bdb62de62349b6b321bb705e25f4be/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:46:55.428168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098ef08-8522-4e8f-a0f4-703708080f39 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:55.473000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 22/43] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:46:55.544869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8098ef08-8522-4e8f-a0f4-703708080f39 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:55.581830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dffe8-8cfa-4dbd-a624-2b81e32e2d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:55.582531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dffe8-8cfa-4dbd-a624-2b81e32e2d84 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:55.582781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dffe8-8cfa-4dbd-a624-2b81e32e2d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:55.583054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829dffe8-8cfa-4dbd-a624-2b81e32e2d84 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:55.583743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dffe8-8cfa-4dbd-a624-2b81e32e2d84 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:55.585169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829dffe8-8cfa-4dbd-a624-2b81e32e2d84 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:55.608036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 22/44] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Sep 13 06:46:55 2022] POST /identity/v3/roles => generated 223 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:46:55.755003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a5828-51ad-4db7-9d32-1aa0c039515a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:55.815885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:55.858980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419a5828-51ad-4db7-9d32-1aa0c039515a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:55.891878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ade69e-c658-468f-8af9-3bf71784f1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:55.892526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ade69e-c658-468f-8af9-3bf71784f1c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:55.892815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ade69e-c658-468f-8af9-3bf71784f1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:55.893087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ade69e-c658-468f-8af9-3bf71784f1c4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:55.895747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ade69e-c658-468f-8af9-3bf71784f1c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:55.895747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ade69e-c658-468f-8af9-3bf71784f1c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:55.897310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 23/45] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:46:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:55.903093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dffe8-8cfa-4dbd-a624-2b81e32e2d84 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:55.953835 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:56.011137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829dffe8-8cfa-4dbd-a624-2b81e32e2d84 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:56.045523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d104f6a9-e390-459a-ae20-ae59e024b2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:56.046184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104f6a9-e390-459a-ae20-ae59e024b2b9 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:56.046470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104f6a9-e390-459a-ae20-ae59e024b2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:56.046745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d104f6a9-e390-459a-ae20-ae59e024b2b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:56.047407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104f6a9-e390-459a-ae20-ae59e024b2b9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:56.048881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d104f6a9-e390-459a-ae20-ae59e024b2b9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:56.080855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 23/46] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:46:55 2022] POST /identity/v3/projects => generated 312 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:46:57.597060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d1d8-8d0d-4cff-8f90-f9d820a353a4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:57.597641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d1d8-8d0d-4cff-8f90-f9d820a353a4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:57.597641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b6d1d8-8d0d-4cff-8f90-f9d820a353a4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:57.599018 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 24/47] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:46:57.604356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633ee31-0a33-4833-89f6-8eeab8f48ff1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:57.604808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633ee31-0a33-4833-89f6-8eeab8f48ff1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:57.604900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1633ee31-0a33-4833-89f6-8eeab8f48ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:57.624741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1633ee31-0a33-4833-89f6-8eeab8f48ff1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:57.625928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1633ee31-0a33-4833-89f6-8eeab8f48ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:57.695369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 24/48] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:46:57.959973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6d1d8-8d0d-4cff-8f90-f9d820a353a4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:58.068659 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b6d1d8-8d0d-4cff-8f90-f9d820a353a4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:58.096032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:58.107776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e78814f-2cc7-4dfb-afb8-5ea17a2eecb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:58.108498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78814f-2cc7-4dfb-afb8-5ea17a2eecb2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:58.108789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78814f-2cc7-4dfb-afb8-5ea17a2eecb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:58.109135 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e78814f-2cc7-4dfb-afb8-5ea17a2eecb2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:58.127805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e78814f-2cc7-4dfb-afb8-5ea17a2eecb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7e78814f-2cc7-4dfb-afb8-5ea17a2eecb2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:46:58.132911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:46:58.133798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:46:58.134604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 25/49] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:46:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:46:58.140232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633ee31-0a33-4833-89f6-8eeab8f48ff1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:58.169356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:58.221251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee11fa4-d004-40a1-a3b7-d72b38c5de5b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:58.221451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee11fa4-d004-40a1-a3b7-d72b38c5de5b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:58.221720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee11fa4-d004-40a1-a3b7-d72b38c5de5b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:58.223535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 26/50] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:46:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:46:58.228799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faabc7-4dab-47e9-8fc7-de63f024a0c0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:58.229089 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faabc7-4dab-47e9-8fc7-de63f024a0c0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:58.229403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5faabc7-4dab-47e9-8fc7-de63f024a0c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:58.247931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f5faabc7-4dab-47e9-8fc7-de63f024a0c0 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:46:58.248140 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1633ee31-0a33-4833-89f6-8eeab8f48ff1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:58.249128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f5faabc7-4dab-47e9-8fc7-de63f024a0c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:46:58.282699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69869f3-4b6e-4e4d-b7cb-957b6332eebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:58.283313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69869f3-4b6e-4e4d-b7cb-957b6332eebe None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:58.283591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69869f3-4b6e-4e4d-b7cb-957b6332eebe None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:58.283865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69869f3-4b6e-4e4d-b7cb-957b6332eebe None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:58.284542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69869f3-4b6e-4e4d-b7cb-957b6332eebe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:46:58.284833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69869f3-4b6e-4e4d-b7cb-957b6332eebe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:58.286275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69869f3-4b6e-4e4d-b7cb-957b6332eebe None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:58.294358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 25/51] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:46:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:58.300732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69869f3-4b6e-4e4d-b7cb-957b6332eebe None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:58.316187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 27/52] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:46:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:46:58.389712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:58.403306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69869f3-4b6e-4e4d-b7cb-957b6332eebe None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:58.551420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a01a18-e06e-4658-a49f-36a975392f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:58.551943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a01a18-e06e-4658-a49f-36a975392f4f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:58.552203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a01a18-e06e-4658-a49f-36a975392f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:58.552473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a01a18-e06e-4658-a49f-36a975392f4f None admin] PATH_INFO: `/v3/users/d3bdb62de62349b6b321bb705e25f4be` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:58.553634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a01a18-e06e-4658-a49f-36a975392f4f None admin] RBAC: Authorizing `identity:get_user(user_id=d3bdb62de62349b6b321bb705e25f4be)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:58.555116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a01a18-e06e-4658-a49f-36a975392f4f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:58.556901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 26/53] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:46:58 2022] GET /identity/v3/users/d3bdb62de62349b6b321bb705e25f4be => generated 247 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:58.562767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faabc7-4dab-47e9-8fc7-de63f024a0c0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:58.585142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a01a18-e06e-4658-a49f-36a975392f4f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:58.667905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5faabc7-4dab-47e9-8fc7-de63f024a0c0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:58.685432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a01a18-e06e-4658-a49f-36a975392f4f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:58.686825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:58.705282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106f81e-151b-4483-884e-19b9df2d600f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:58.705921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106f81e-151b-4483-884e-19b9df2d600f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:58.706234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106f81e-151b-4483-884e-19b9df2d600f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:58.706506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f106f81e-151b-4483-884e-19b9df2d600f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:58.707626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106f81e-151b-4483-884e-19b9df2d600f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:58.709096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f106f81e-151b-4483-884e-19b9df2d600f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:58.710673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 28/54] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:46:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:58.716524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106f81e-151b-4483-884e-19b9df2d600f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:58.721408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556826-e5c4-4bb8-a394-1ee80d636358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:58.722045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556826-e5c4-4bb8-a394-1ee80d636358 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:58.722363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556826-e5c4-4bb8-a394-1ee80d636358 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:58.722633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8556826-e5c4-4bb8-a394-1ee80d636358 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:58.723776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8556826-e5c4-4bb8-a394-1ee80d636358 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:58.725218 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8556826-e5c4-4bb8-a394-1ee80d636358 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:58.726810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 27/55] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:46:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:46:58.732146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556826-e5c4-4bb8-a394-1ee80d636358 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:46:58.757833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:46:58.821103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f106f81e-151b-4483-884e-19b9df2d600f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:58.835180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8556826-e5c4-4bb8-a394-1ee80d636358 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:46:58.857782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520db144-7b30-4eca-be1e-79ee3653a2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:58.858319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520db144-7b30-4eca-be1e-79ee3653a2db None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:58.858600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520db144-7b30-4eca-be1e-79ee3653a2db None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:58.858868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520db144-7b30-4eca-be1e-79ee3653a2db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:58.859591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520db144-7b30-4eca-be1e-79ee3653a2db None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.domain.id=default, user.id=d3bdb62de62349b6b321bb705e25f4be) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:46:58.859913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520db144-7b30-4eca-be1e-79ee3653a2db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:58.861348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520db144-7b30-4eca-be1e-79ee3653a2db None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:58.867057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 29/56] 10.222.0.25 () {56 vars in 1322 bytes} [Tue Sep 13 06:46:58 2022] GET /identity/v3/role_assignments?user.id=d3bdb62de62349b6b321bb705e25f4be&scope.domain.id=default&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 530 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:46:58.871789 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf076b25-3404-454e-b2bd-56ffc6efc8bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:46:58.872426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf076b25-3404-454e-b2bd-56ffc6efc8bb None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:46:58.872706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf076b25-3404-454e-b2bd-56ffc6efc8bb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:46:58.872986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf076b25-3404-454e-b2bd-56ffc6efc8bb None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:46:58.873713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf076b25-3404-454e-b2bd-56ffc6efc8bb None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:46:58.875208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf076b25-3404-454e-b2bd-56ffc6efc8bb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:46:58.907614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 28/57] 10.222.0.25 () {60 vars in 1140 bytes} [Tue Sep 13 06:46:58 2022] POST /identity/v3/projects => generated 298 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:47:01.141098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b157490-cd53-4625-b934-308abc5b9c3f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:01.141950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b157490-cd53-4625-b934-308abc5b9c3f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:01.141950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b157490-cd53-4625-b934-308abc5b9c3f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:01.144177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 30/58] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:01.151080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1698d-b8e1-4b6a-a8aa-fd16f3f0c5b4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:01.151799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1698d-b8e1-4b6a-a8aa-fd16f3f0c5b4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:01.151942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b1698d-b8e1-4b6a-a8aa-fd16f3f0c5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:01.177681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a4b1698d-b8e1-4b6a-a8aa-fd16f3f0c5b4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:01.178917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a4b1698d-b8e1-4b6a-a8aa-fd16f3f0c5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:01.250421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 29/59] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:01.515229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b157490-cd53-4625-b934-308abc5b9c3f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:01.574800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:01.629609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b157490-cd53-4625-b934-308abc5b9c3f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:01.664291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50404a85-efe9-40e5-a4e1-579e5b249c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:01.664921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50404a85-efe9-40e5-a4e1-579e5b249c21 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:01.665212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50404a85-efe9-40e5-a4e1-579e5b249c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:01.665490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50404a85-efe9-40e5-a4e1-579e5b249c21 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:01.666688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50404a85-efe9-40e5-a4e1-579e5b249c21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:01.668158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50404a85-efe9-40e5-a4e1-579e5b249c21 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:01.669731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 31/60] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:01.675633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1698d-b8e1-4b6a-a8aa-fd16f3f0c5b4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:01.684498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:01.787647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b1698d-b8e1-4b6a-a8aa-fd16f3f0c5b4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:01.823900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f4c9416-157e-4916-86b5-c6b00b811b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:01.824519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c9416-157e-4916-86b5-c6b00b811b0d None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:01.824805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c9416-157e-4916-86b5-c6b00b811b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:01.825078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f4c9416-157e-4916-86b5-c6b00b811b0d None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:01.825722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c9416-157e-4916-86b5-c6b00b811b0d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:01.827752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f4c9416-157e-4916-86b5-c6b00b811b0d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:01.858978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 30/61] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:47:01 2022] POST /identity/v3/users => generated 275 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:47:05.284244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab597dd-3280-4de4-b3f5-6bb29d305864 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:05.284788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab597dd-3280-4de4-b3f5-6bb29d305864 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:05.284788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab597dd-3280-4de4-b3f5-6bb29d305864 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:05.286182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 32/62] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:05.292010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243649-b4e2-4eff-a9d8-9fabac004931 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:05.292847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243649-b4e2-4eff-a9d8-9fabac004931 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:05.293273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd243649-b4e2-4eff-a9d8-9fabac004931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:05.327143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cd243649-b4e2-4eff-a9d8-9fabac004931 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:05.328597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cd243649-b4e2-4eff-a9d8-9fabac004931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:05.333991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013b28c-76ce-45b7-803d-599f9fc3a4cb None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:05.334308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013b28c-76ce-45b7-803d-599f9fc3a4cb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:05.334819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3013b28c-76ce-45b7-803d-599f9fc3a4cb None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:05.336404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 33/63] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:05.341844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceadfc-c7ac-4413-8c6a-a03598c30a06 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:05.343018 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceadfc-c7ac-4413-8c6a-a03598c30a06 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:05.343517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ceadfc-c7ac-4413-8c6a-a03598c30a06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:05.379171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-74ceadfc-c7ac-4413-8c6a-a03598c30a06 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:05.380386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-74ceadfc-c7ac-4413-8c6a-a03598c30a06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:05.428617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:05.441098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 31/64] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:05.443006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd25b09-4fc5-4f55-a874-6deb68925919 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:05.443907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd25b09-4fc5-4f55-a874-6deb68925919 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:05.444399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd25b09-4fc5-4f55-a874-6deb68925919 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:05.446031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 32/65] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:05.456376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa03fd-969d-477a-930c-5c9a64f96263 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:05.457019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa03fd-969d-477a-930c-5c9a64f96263 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:05.457487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa03fd-969d-477a-930c-5c9a64f96263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:05.479685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3daa03fd-969d-477a-930c-5c9a64f96263 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:05.481749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3daa03fd-969d-477a-930c-5c9a64f96263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:05.539944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 34/66] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:05.575840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce21e6-a417-4442-acb8-65aaba9bf142 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:05.576462 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce21e6-a417-4442-acb8-65aaba9bf142 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:05.576462 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ce21e6-a417-4442-acb8-65aaba9bf142 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:05.577789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 35/67] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:47:05.583290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7b22-147f-44ab-8ef2-e8a80c492397 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:05.583573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7b22-147f-44ab-8ef2-e8a80c492397 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:05.583847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aa7b22-147f-44ab-8ef2-e8a80c492397 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:05.584950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 33/68] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:05.613739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-90aa7b22-147f-44ab-8ef2-e8a80c492397 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:05.614979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-90aa7b22-147f-44ab-8ef2-e8a80c492397 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:05.670163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:05.701059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 36/69] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Sep 13 06:47:05.839817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa03fd-969d-477a-930c-5c9a64f96263 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:05.870277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa7b22-147f-44ab-8ef2-e8a80c492397 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:05.981040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa03fd-969d-477a-930c-5c9a64f96263 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.000547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aa7b22-147f-44ab-8ef2-e8a80c492397 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.005014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:06.017931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18584790-1fd7-49da-ad54-af268e7c659c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.018924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584790-1fd7-49da-ad54-af268e7c659c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.019398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584790-1fd7-49da-ad54-af268e7c659c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.019858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18584790-1fd7-49da-ad54-af268e7c659c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.031628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18584790-1fd7-49da-ad54-af268e7c659c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-18584790-1fd7-49da-ad54-af268e7c659c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:06.036819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:06.037645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:06.038449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:06.039017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5ae3d-1dd6-4db2-97ed-fbf574bf08a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.039017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ae3d-1dd6-4db2-97ed-fbf574bf08a9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.039017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ae3d-1dd6-4db2-97ed-fbf574bf08a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.039017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa5ae3d-1dd6-4db2-97ed-fbf574bf08a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.041058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 34/70] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:06.042144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18584790-1fd7-49da-ad54-af268e7c659c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.049985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa5ae3d-1dd6-4db2-97ed-fbf574bf08a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2aa5ae3d-1dd6-4db2-97ed-fbf574bf08a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:06.055112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:06.055963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:06.056725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 37/71] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:47:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:47:06.057345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5ae3d-1dd6-4db2-97ed-fbf574bf08a9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.097602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:06.148834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18584790-1fd7-49da-ad54-af268e7c659c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.161952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa5ae3d-1dd6-4db2-97ed-fbf574bf08a9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.184254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77be71e-500e-43ab-89da-c5e40575aa68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.184905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77be71e-500e-43ab-89da-c5e40575aa68 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.185188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77be71e-500e-43ab-89da-c5e40575aa68 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.185477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77be71e-500e-43ab-89da-c5e40575aa68 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.195269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a774c08-b665-49fe-a6c4-d56fda24f148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.195557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77be71e-500e-43ab-89da-c5e40575aa68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:06.195884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a774c08-b665-49fe-a6c4-d56fda24f148 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.196172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a774c08-b665-49fe-a6c4-d56fda24f148 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.196445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a774c08-b665-49fe-a6c4-d56fda24f148 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.197109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a774c08-b665-49fe-a6c4-d56fda24f148 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:06.197412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a774c08-b665-49fe-a6c4-d56fda24f148 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:06.198848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a774c08-b665-49fe-a6c4-d56fda24f148 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d77be71e-500e-43ab-89da-c5e40575aa68 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:06.200486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:06.201381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:06.202182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 35/72] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:06.202823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77be71e-500e-43ab-89da-c5e40575aa68 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.209499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 38/73] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:06.210339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a774c08-b665-49fe-a6c4-d56fda24f148 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.247609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:06.302738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:06.305236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77be71e-500e-43ab-89da-c5e40575aa68 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.306490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a774c08-b665-49fe-a6c4-d56fda24f148 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.343337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148a1f77-f695-4471-b4cf-67f8dcd95ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.343566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cfe93-29b8-4f05-b893-229c3a7b421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.343975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a1f77-f695-4471-b4cf-67f8dcd95ee9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.344205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cfe93-29b8-4f05-b893-229c3a7b421d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.344326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a1f77-f695-4471-b4cf-67f8dcd95ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.344491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cfe93-29b8-4f05-b893-229c3a7b421d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.344585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148a1f77-f695-4471-b4cf-67f8dcd95ee9 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.344767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71cfe93-29b8-4f05-b893-229c3a7b421d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.345160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a1f77-f695-4471-b4cf-67f8dcd95ee9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:06.345465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a1f77-f695-4471-b4cf-67f8dcd95ee9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:06.346895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148a1f77-f695-4471-b4cf-67f8dcd95ee9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:06.354777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 39/74] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:06.355863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c691b6-9d74-44dd-8cc3-696ab1380b75 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.356130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c691b6-9d74-44dd-8cc3-696ab1380b75 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.356400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c691b6-9d74-44dd-8cc3-696ab1380b75 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.357745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 40/75] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:06 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:06.359683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71cfe93-29b8-4f05-b893-229c3a7b421d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:06.362476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c691b6-9d74-44dd-8cc3-696ab1380b75 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f71cfe93-29b8-4f05-b893-229c3a7b421d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:06.364692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:06.365592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:06.366379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 36/76] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:06.367127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cfe93-29b8-4f05-b893-229c3a7b421d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.469154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71cfe93-29b8-4f05-b893-229c3a7b421d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.470746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:06.482169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c691b6-9d74-44dd-8cc3-696ab1380b75 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.505397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5afe3-51b2-446c-9364-4fe480cbb384 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.506060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5afe3-51b2-446c-9364-4fe480cbb384 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.506334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5afe3-51b2-446c-9364-4fe480cbb384 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.506631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e5afe3-51b2-446c-9364-4fe480cbb384 None admin] PATH_INFO: `/v3/users/a0286a556e15432dbc2bd4ac3f2e801a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.517086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb606d6-7781-4552-896f-695f3f4f40a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.517703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb606d6-7781-4552-896f-695f3f4f40a4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.517982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb606d6-7781-4552-896f-695f3f4f40a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.518291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb606d6-7781-4552-896f-695f3f4f40a4 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.518955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb606d6-7781-4552-896f-695f3f4f40a4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:06.519262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb606d6-7781-4552-896f-695f3f4f40a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:06.520702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb606d6-7781-4552-896f-695f3f4f40a4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:06.521941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5afe3-51b2-446c-9364-4fe480cbb384 None admin] RBAC: Authorizing `identity:get_user(user_id=a0286a556e15432dbc2bd4ac3f2e801a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:06.523448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e5afe3-51b2-446c-9364-4fe480cbb384 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:06.525056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 37/77] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/users/a0286a556e15432dbc2bd4ac3f2e801a => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:06.526002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5afe3-51b2-446c-9364-4fe480cbb384 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.530155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 41/78] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:06.532270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e9462-b453-44f8-a2a6-cd557f15ef26 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.532561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e9462-b453-44f8-a2a6-cd557f15ef26 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.532845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629e9462-b453-44f8-a2a6-cd557f15ef26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.551356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-629e9462-b453-44f8-a2a6-cd557f15ef26 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:06.552585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-629e9462-b453-44f8-a2a6-cd557f15ef26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:06.570637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:06.624600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 42/79] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:06.625456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e9462-b453-44f8-a2a6-cd557f15ef26 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.637468 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e5afe3-51b2-446c-9364-4fe480cbb384 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.675166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006058e3-f20c-469d-ad38-18d8b3823d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.675954 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006058e3-f20c-469d-ad38-18d8b3823d86 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.676292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006058e3-f20c-469d-ad38-18d8b3823d86 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.676572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006058e3-f20c-469d-ad38-18d8b3823d86 None admin] PATH_INFO: `/v3/users/d3bdb62de62349b6b321bb705e25f4be` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.677764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006058e3-f20c-469d-ad38-18d8b3823d86 None admin] RBAC: Authorizing `identity:get_user(user_id=d3bdb62de62349b6b321bb705e25f4be)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:06.679509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006058e3-f20c-469d-ad38-18d8b3823d86 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:06.685959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 38/80] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/users/d3bdb62de62349b6b321bb705e25f4be => generated 247 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:06.685959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006058e3-f20c-469d-ad38-18d8b3823d86 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.714702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:06.727804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629e9462-b453-44f8-a2a6-cd557f15ef26 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.762120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b5921-d4c6-4d9f-844d-ff4df13ca3e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.762748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b5921-d4c6-4d9f-844d-ff4df13ca3e3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.763038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b5921-d4c6-4d9f-844d-ff4df13ca3e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.763312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b5921-d4c6-4d9f-844d-ff4df13ca3e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.764069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b5921-d4c6-4d9f-844d-ff4df13ca3e3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:06.764468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b5921-d4c6-4d9f-844d-ff4df13ca3e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:06.765892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b5921-d4c6-4d9f-844d-ff4df13ca3e3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:06.773927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 43/81] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:06.773927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b5921-d4c6-4d9f-844d-ff4df13ca3e3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.791209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:06.802988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006058e3-f20c-469d-ad38-18d8b3823d86 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.840356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79ec2b-57ab-496c-ad59-835f2269dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.840985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79ec2b-57ab-496c-ad59-835f2269dc64 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.841272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79ec2b-57ab-496c-ad59-835f2269dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.841552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e79ec2b-57ab-496c-ad59-835f2269dc64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.879735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b5921-d4c6-4d9f-844d-ff4df13ca3e3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:06.911644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79ec2b-57ab-496c-ad59-835f2269dc64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:06.913150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e79ec2b-57ab-496c-ad59-835f2269dc64 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:06.913719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8a856-c72c-4cf7-b8ec-fa1f276ff855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:06.914396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8a856-c72c-4cf7-b8ec-fa1f276ff855 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:06.914677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8a856-c72c-4cf7-b8ec-fa1f276ff855 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:06.914970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f8a856-c72c-4cf7-b8ec-fa1f276ff855 None admin] PATH_INFO: `/v3/users/a0286a556e15432dbc2bd4ac3f2e801a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:06.916068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8a856-c72c-4cf7-b8ec-fa1f276ff855 None admin] RBAC: Authorizing `identity:get_user(user_id=a0286a556e15432dbc2bd4ac3f2e801a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:06.917497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f8a856-c72c-4cf7-b8ec-fa1f276ff855 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:06.919110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 44/82] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/users/a0286a556e15432dbc2bd4ac3f2e801a => generated 275 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:06.919975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8a856-c72c-4cf7-b8ec-fa1f276ff855 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:06.944613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.002196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.021733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 39/83] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:07.021733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79ec2b-57ab-496c-ad59-835f2269dc64 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.026158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f8a856-c72c-4cf7-b8ec-fa1f276ff855 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:07.061028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7ba62-239b-40b4-8625-f23214834396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:07.061823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7ba62-239b-40b4-8625-f23214834396 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:07.062174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7ba62-239b-40b4-8625-f23214834396 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:07.062493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d7ba62-239b-40b4-8625-f23214834396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:07.117456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e79ec2b-57ab-496c-ad59-835f2269dc64 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:07.131105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7ba62-239b-40b4-8625-f23214834396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:07.132544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d7ba62-239b-40b4-8625-f23214834396 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:07.143113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.153032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db07e4b-7f66-47dc-a297-37a9a217c1f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:07.153661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db07e4b-7f66-47dc-a297-37a9a217c1f5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:07.153948 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db07e4b-7f66-47dc-a297-37a9a217c1f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:07.154307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db07e4b-7f66-47dc-a297-37a9a217c1f5 None admin] PATH_INFO: `/v3/users/a0286a556e15432dbc2bd4ac3f2e801a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:07.155722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db07e4b-7f66-47dc-a297-37a9a217c1f5 None admin] RBAC: Authorizing `identity:get_user(user_id=a0286a556e15432dbc2bd4ac3f2e801a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:07.157181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db07e4b-7f66-47dc-a297-37a9a217c1f5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:07.158782 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 40/84] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:07 2022] GET /identity/v3/users/a0286a556e15432dbc2bd4ac3f2e801a => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:07.159540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db07e4b-7f66-47dc-a297-37a9a217c1f5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.217103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.237430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 45/85] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:07.238859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7ba62-239b-40b4-8625-f23214834396 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.262832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db07e4b-7f66-47dc-a297-37a9a217c1f5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:07.298384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b45eb7-50b2-4f5d-bb9e-26353166a632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:07.299022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b45eb7-50b2-4f5d-bb9e-26353166a632 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:07.299302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b45eb7-50b2-4f5d-bb9e-26353166a632 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:07.299572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b45eb7-50b2-4f5d-bb9e-26353166a632 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:07.300704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b45eb7-50b2-4f5d-bb9e-26353166a632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:07.302198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b45eb7-50b2-4f5d-bb9e-26353166a632 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:07.306116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 41/86] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:07.306116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b45eb7-50b2-4f5d-bb9e-26353166a632 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.338330 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d7ba62-239b-40b4-8625-f23214834396 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:07.369075 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.371473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2f917-49ef-4c90-a35a-386a6ae1a83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:07.372091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2f917-49ef-4c90-a35a-386a6ae1a83d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:07.372369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2f917-49ef-4c90-a35a-386a6ae1a83d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:07.372640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b2f917-49ef-4c90-a35a-386a6ae1a83d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:07.408031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b45eb7-50b2-4f5d-bb9e-26353166a632 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:07.421561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.440477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2f917-49ef-4c90-a35a-386a6ae1a83d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:07.441966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b2f917-49ef-4c90-a35a-386a6ae1a83d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:07.442328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855092a-5e5f-4e02-a318-5183df6b295c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:07.442966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855092a-5e5f-4e02-a318-5183df6b295c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:07.443254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855092a-5e5f-4e02-a318-5183df6b295c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:07.443531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c855092a-5e5f-4e02-a318-5183df6b295c None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:07.444681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855092a-5e5f-4e02-a318-5183df6b295c None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:07.446201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c855092a-5e5f-4e02-a318-5183df6b295c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:07.447852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 42/87] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:07 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:07.448613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855092a-5e5f-4e02-a318-5183df6b295c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.544759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c855092a-5e5f-4e02-a318-5183df6b295c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:07.545350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 46/88] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:07.546386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2f917-49ef-4c90-a35a-386a6ae1a83d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.574721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.580319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb9aab-64d4-4407-9d95-bd0894ea44f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:07.580941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb9aab-64d4-4407-9d95-bd0894ea44f4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:07.581220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb9aab-64d4-4407-9d95-bd0894ea44f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:07.581495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb9aab-64d4-4407-9d95-bd0894ea44f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:07.629632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.648831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b2f917-49ef-4c90-a35a-386a6ae1a83d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:07.651239 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb9aab-64d4-4407-9d95-bd0894ea44f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:07.652744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bb9aab-64d4-4407-9d95-bd0894ea44f4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:07.682284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77cf31-5e9d-4ce4-95f2-466ac9af4f71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:07.682845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77cf31-5e9d-4ce4-95f2-466ac9af4f71 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:07.683472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77cf31-5e9d-4ce4-95f2-466ac9af4f71 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:07.683744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f77cf31-5e9d-4ce4-95f2-466ac9af4f71 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:07.684877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77cf31-5e9d-4ce4-95f2-466ac9af4f71 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:07.686668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f77cf31-5e9d-4ce4-95f2-466ac9af4f71 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:07.688303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 47/89] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:07 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:07.689042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77cf31-5e9d-4ce4-95f2-466ac9af4f71 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.751104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 43/90] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:07.752227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb9aab-64d4-4407-9d95-bd0894ea44f4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.782743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.785794 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f77cf31-5e9d-4ce4-95f2-466ac9af4f71 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:07.818955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57e184-ac16-4900-8bb4-69592af92945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:07.819568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57e184-ac16-4900-8bb4-69592af92945 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:07.819848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57e184-ac16-4900-8bb4-69592af92945 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:07.820122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57e184-ac16-4900-8bb4-69592af92945 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:07.820771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57e184-ac16-4900-8bb4-69592af92945 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:07.822260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57e184-ac16-4900-8bb4-69592af92945 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:07.850610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 48/91] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:47:07 2022] POST /identity/v3/users => generated 289 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:47:07.852740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb9aab-64d4-4407-9d95-bd0894ea44f4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:07.855124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57e184-ac16-4900-8bb4-69592af92945 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.864836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:07.895934 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead5927-9741-4f42-8680-359d24577b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:07.896702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead5927-9741-4f42-8680-359d24577b3b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:07.897812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead5927-9741-4f42-8680-359d24577b3b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:07.897812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ead5927-9741-4f42-8680-359d24577b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:07.898136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead5927-9741-4f42-8680-359d24577b3b None admin] RBAC: Adding query filter params (role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=ab8379e77b244774a8bf8329866e9014, user.id=a0286a556e15432dbc2bd4ac3f2e801a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:07.898451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead5927-9741-4f42-8680-359d24577b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:07.899927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ead5927-9741-4f42-8680-359d24577b3b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:07.918291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 44/92] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:07 2022] GET /identity/v3/role_assignments?user.id=a0286a556e15432dbc2bd4ac3f2e801a&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=c622170611454f18bf01aa658a8cc06e => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:07.918291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead5927-9741-4f42-8680-359d24577b3b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:07.991397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57e184-ac16-4900-8bb4-69592af92945 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:08.034938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f73bde-7eea-4df4-ac50-daebb9366735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:08.036108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:08.036817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f73bde-7eea-4df4-ac50-daebb9366735 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:08.037114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f73bde-7eea-4df4-ac50-daebb9366735 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:08.037394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f73bde-7eea-4df4-ac50-daebb9366735 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:08.038990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f73bde-7eea-4df4-ac50-daebb9366735 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:08.040528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f73bde-7eea-4df4-ac50-daebb9366735 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:08.043886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 49/93] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:07 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:08.045073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f73bde-7eea-4df4-ac50-daebb9366735 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:08.045806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ead5927-9741-4f42-8680-359d24577b3b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:08.096851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386198f3-8ce0-4310-840e-4a4ebe403174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:08.097812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386198f3-8ce0-4310-840e-4a4ebe403174 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:08.098307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386198f3-8ce0-4310-840e-4a4ebe403174 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:08.098750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386198f3-8ce0-4310-840e-4a4ebe403174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:08.099704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386198f3-8ce0-4310-840e-4a4ebe403174 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=ab8379e77b244774a8bf8329866e9014, user.id=d3bdb62de62349b6b321bb705e25f4be) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:08.100608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386198f3-8ce0-4310-840e-4a4ebe403174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:08.101499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386198f3-8ce0-4310-840e-4a4ebe403174 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:08.113299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 45/94] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:07 2022] GET /identity/v3/role_assignments?user.id=d3bdb62de62349b6b321bb705e25f4be&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:08.113988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386198f3-8ce0-4310-840e-4a4ebe403174 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:08.162426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:08.225766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f73bde-7eea-4df4-ac50-daebb9366735 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:08.255318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386198f3-8ce0-4310-840e-4a4ebe403174 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:08.275160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c9877-cbd8-4138-b8e4-b6d918af61e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:08.276166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c9877-cbd8-4138-b8e4-b6d918af61e1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:08.276509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c9877-cbd8-4138-b8e4-b6d918af61e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:08.276857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c9877-cbd8-4138-b8e4-b6d918af61e1 None admin] PATH_INFO: `/v3/projects/913b7cf334eb42248201177cfff59a6b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:08.279620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c9877-cbd8-4138-b8e4-b6d918af61e1 None admin] RBAC: Authorizing `identity:get_project(project_id=913b7cf334eb42248201177cfff59a6b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:08.280834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293c9877-cbd8-4138-b8e4-b6d918af61e1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:08.283624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 50/95] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:08 2022] GET /identity/v3/projects/913b7cf334eb42248201177cfff59a6b => generated 312 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:08.291585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c9877-cbd8-4138-b8e4-b6d918af61e1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:08.303825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397c06b8-b559-496f-9344-2706501991ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:08.304825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c06b8-b559-496f-9344-2706501991ba None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:08.305279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c06b8-b559-496f-9344-2706501991ba None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:08.305777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397c06b8-b559-496f-9344-2706501991ba None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:08.306872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c06b8-b559-496f-9344-2706501991ba None admin] RBAC: Adding query filter params (role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=ab8379e77b244774a8bf8329866e9014, user.id=a0286a556e15432dbc2bd4ac3f2e801a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:08.307821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c06b8-b559-496f-9344-2706501991ba None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:08.309917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397c06b8-b559-496f-9344-2706501991ba None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:08.318736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 46/96] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:08 2022] GET /identity/v3/role_assignments?user.id=a0286a556e15432dbc2bd4ac3f2e801a&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=2d59fcad66104464bbc848ef605f743e => generated 259 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:08.421266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293c9877-cbd8-4138-b8e4-b6d918af61e1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:08.464648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5943375b-083f-4a14-87f1-56deafabf718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:08.465296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943375b-083f-4a14-87f1-56deafabf718 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:08.465628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943375b-083f-4a14-87f1-56deafabf718 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:08.465906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5943375b-083f-4a14-87f1-56deafabf718 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:08.466716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943375b-083f-4a14-87f1-56deafabf718 None admin] RBAC: Adding query filter params (role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=913b7cf334eb42248201177cfff59a6b, user.id=a0286a556e15432dbc2bd4ac3f2e801a) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:08.467094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943375b-083f-4a14-87f1-56deafabf718 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:08.468611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5943375b-083f-4a14-87f1-56deafabf718 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:08.476076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 51/97] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:08 2022] GET /identity/v3/role_assignments?user.id=a0286a556e15432dbc2bd4ac3f2e801a&scope.project.id=913b7cf334eb42248201177cfff59a6b&role.id=2d59fcad66104464bbc848ef605f743e => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:10.546888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec64c8-1e45-443a-b247-9cfac2e82520 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:10.547928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec64c8-1e45-443a-b247-9cfac2e82520 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:10.548411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec64c8-1e45-443a-b247-9cfac2e82520 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:10.549966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 47/98] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:47:10.562121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e688c99-1c10-4861-8e20-c622ad062620 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:10.563068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e688c99-1c10-4861-8e20-c622ad062620 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:10.563552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e688c99-1c10-4861-8e20-c622ad062620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:10.582158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:10.588110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4e688c99-1c10-4861-8e20-c622ad062620 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:10.589380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e688c99-1c10-4861-8e20-c622ad062620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:10.698789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 52/99] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:11.002456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d988c-2574-4584-867b-4616620f6ea5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.002976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d988c-2574-4584-867b-4616620f6ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.003392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1d988c-2574-4584-867b-4616620f6ea5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.004894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 48/100] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:11.010382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c95e4-db0d-4370-ac8d-23316b1366cd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.010829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c95e4-db0d-4370-ac8d-23316b1366cd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.011246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c95e4-db0d-4370-ac8d-23316b1366cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.033793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-445c95e4-db0d-4370-ac8d-23316b1366cd None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:11.035355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-445c95e4-db0d-4370-ac8d-23316b1366cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:11.130513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 53/101] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:11.295797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d988c-2574-4584-867b-4616620f6ea5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:11.328981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:11.344946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78070d60-e28f-4cc1-84b4-ca5e673173f2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.345267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78070d60-e28f-4cc1-84b4-ca5e673173f2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78070d60-e28f-4cc1-84b4-ca5e673173f2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.347138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 54/102] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:47:11.359349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb897aa-cbe6-49a0-9d27-c73c621940ac None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.359655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb897aa-cbe6-49a0-9d27-c73c621940ac None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.359903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb897aa-cbe6-49a0-9d27-c73c621940ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.395147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4bb897aa-cbe6-49a0-9d27-c73c621940ac None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:11.396358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4bb897aa-cbe6-49a0-9d27-c73c621940ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:11.488817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1d988c-2574-4584-867b-4616620f6ea5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:11.533385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024412a-4c8c-4934-a971-b66b4f86aa25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:11.534296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024412a-4c8c-4934-a971-b66b4f86aa25 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.534716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024412a-4c8c-4934-a971-b66b4f86aa25 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.535156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4024412a-4c8c-4934-a971-b66b4f86aa25 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.547037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:11.551905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 55/103] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:11.553508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff2181-55c5-4aeb-9deb-a801d339c589 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.554162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff2181-55c5-4aeb-9deb-a801d339c589 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.554871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feff2181-55c5-4aeb-9deb-a801d339c589 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.556382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 56/104] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:11.565703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4024412a-4c8c-4934-a971-b66b4f86aa25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:11.567472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276c3fd-34ec-45b1-8755-eb76c8cfba38 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.567859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276c3fd-34ec-45b1-8755-eb76c8cfba38 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.568016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276c3fd-34ec-45b1-8755-eb76c8cfba38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4024412a-4c8c-4934-a971-b66b4f86aa25 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:11.572762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:11.573619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:11.574419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:11.577611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 49/105] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:11.583833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024412a-4c8c-4934-a971-b66b4f86aa25 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:11.593662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f276c3fd-34ec-45b1-8755-eb76c8cfba38 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:11.603089 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f276c3fd-34ec-45b1-8755-eb76c8cfba38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:11.682522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 57/106] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:11.716664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276c3fd-34ec-45b1-8755-eb76c8cfba38 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:11.734557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:11.750705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4024412a-4c8c-4934-a971-b66b4f86aa25 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:11.816797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eddf73f-b4b9-4e4e-be77-4d3da6b569bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:11.817700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddf73f-b4b9-4e4e-be77-4d3da6b569bf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.818143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddf73f-b4b9-4e4e-be77-4d3da6b569bf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.818557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eddf73f-b4b9-4e4e-be77-4d3da6b569bf None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.819394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddf73f-b4b9-4e4e-be77-4d3da6b569bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:11.819837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddf73f-b4b9-4e4e-be77-4d3da6b569bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:11.821442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eddf73f-b4b9-4e4e-be77-4d3da6b569bf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:11.824205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276c3fd-34ec-45b1-8755-eb76c8cfba38 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:11.848133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 50/107] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:11.849642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bd778-b347-4e08-8133-949d0c2bbea0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.849816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bd778-b347-4e08-8133-949d0c2bbea0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.850059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16bd778-b347-4e08-8133-949d0c2bbea0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.851502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 51/108] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:11.853357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bd778-b347-4e08-8133-949d0c2bbea0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:11.853834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:11.865895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9bd9bb-925d-46b2-bc49-8dd5ec4c4c6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:11.867766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bd9bb-925d-46b2-bc49-8dd5ec4c4c6f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.868248 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bd9bb-925d-46b2-bc49-8dd5ec4c4c6f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.868711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9bd9bb-925d-46b2-bc49-8dd5ec4c4c6f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.879939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9bd9bb-925d-46b2-bc49-8dd5ec4c4c6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7f9bd9bb-925d-46b2-bc49-8dd5ec4c4c6f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:11.885291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:11.886154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:11.886928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:11.889639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 58/109] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:47:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:11.895850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d714b2-3e44-4ecd-85fc-f9aca46d25ba None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:11.896420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d714b2-3e44-4ecd-85fc-f9aca46d25ba None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:11.896908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d714b2-3e44-4ecd-85fc-f9aca46d25ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:11.916101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-92d714b2-3e44-4ecd-85fc-f9aca46d25ba None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:11.917633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-92d714b2-3e44-4ecd-85fc-f9aca46d25ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:11.989240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 59/110] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:11.990440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d714b2-3e44-4ecd-85fc-f9aca46d25ba None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.049859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16bd778-b347-4e08-8133-949d0c2bbea0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.121564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdd1bd-ebab-4c24-bef6-2197fb1f7a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.122968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdd1bd-ebab-4c24-bef6-2197fb1f7a6a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.123536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdd1bd-ebab-4c24-bef6-2197fb1f7a6a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.124819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fdd1bd-ebab-4c24-bef6-2197fb1f7a6a None admin] PATH_INFO: `/v3/users/a0286a556e15432dbc2bd4ac3f2e801a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.126296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdd1bd-ebab-4c24-bef6-2197fb1f7a6a None admin] RBAC: Authorizing `identity:get_user(user_id=a0286a556e15432dbc2bd4ac3f2e801a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:12.128009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fdd1bd-ebab-4c24-bef6-2197fb1f7a6a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:12.130613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 52/111] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:11 2022] GET /identity/v3/users/a0286a556e15432dbc2bd4ac3f2e801a => generated 275 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:12.131642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdd1bd-ebab-4c24-bef6-2197fb1f7a6a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.150008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:12.166887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d714b2-3e44-4ecd-85fc-f9aca46d25ba None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.184592 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:12.205081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e280d-e778-4fc7-a171-160e93617d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.205864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e280d-e778-4fc7-a171-160e93617d93 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.206180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e280d-e778-4fc7-a171-160e93617d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.206463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8e280d-e778-4fc7-a171-160e93617d93 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.207144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e280d-e778-4fc7-a171-160e93617d93 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:12.207450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e280d-e778-4fc7-a171-160e93617d93 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:12.208870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8e280d-e778-4fc7-a171-160e93617d93 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:12.218205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 60/112] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:12.223263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e280d-e778-4fc7-a171-160e93617d93 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.248711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fdd1bd-ebab-4c24-bef6-2197fb1f7a6a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.285019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03524f-f6c2-40fc-b7d4-e4183f13f1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.285662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03524f-f6c2-40fc-b7d4-e4183f13f1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.285951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03524f-f6c2-40fc-b7d4-e4183f13f1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.286261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c03524f-f6c2-40fc-b7d4-e4183f13f1d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.287357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03524f-f6c2-40fc-b7d4-e4183f13f1d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:12.288820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c03524f-f6c2-40fc-b7d4-e4183f13f1d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:12.290884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 53/113] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:12.291177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03524f-f6c2-40fc-b7d4-e4183f13f1d2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.324073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8e280d-e778-4fc7-a171-160e93617d93 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.357234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db47b7b3-18c5-4b1c-8b73-eeccbee603c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.357981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47b7b3-18c5-4b1c-8b73-eeccbee603c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.358185 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47b7b3-18c5-4b1c-8b73-eeccbee603c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.358465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db47b7b3-18c5-4b1c-8b73-eeccbee603c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.370132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db47b7b3-18c5-4b1c-8b73-eeccbee603c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:12.375296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-db47b7b3-18c5-4b1c-8b73-eeccbee603c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:12.375425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:12.376397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:12.377434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 61/114] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:12.378213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107b4e4-4afb-4573-a66a-3d013d0aa4ba None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.378213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107b4e4-4afb-4573-a66a-3d013d0aa4ba None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.378906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107b4e4-4afb-4573-a66a-3d013d0aa4ba None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.382136 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 62/115] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:12.382136 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107b4e4-4afb-4573-a66a-3d013d0aa4ba None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.398439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c03524f-f6c2-40fc-b7d4-e4183f13f1d2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.424543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:12.433838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74d19f-e85e-44da-8b76-321bfa84fce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.434531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74d19f-e85e-44da-8b76-321bfa84fce7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.434809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74d19f-e85e-44da-8b76-321bfa84fce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.435079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b74d19f-e85e-44da-8b76-321bfa84fce7 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.436329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74d19f-e85e-44da-8b76-321bfa84fce7 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:12.437803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b74d19f-e85e-44da-8b76-321bfa84fce7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:12.439476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 54/116] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:12 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:12.440232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74d19f-e85e-44da-8b76-321bfa84fce7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.491766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107b4e4-4afb-4573-a66a-3d013d0aa4ba None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.526992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdba9ef-07e1-40b0-bf15-10f88c571d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.527740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdba9ef-07e1-40b0-bf15-10f88c571d34 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.528091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdba9ef-07e1-40b0-bf15-10f88c571d34 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.528438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdba9ef-07e1-40b0-bf15-10f88c571d34 None admin] PATH_INFO: `/v3/users/d3bdb62de62349b6b321bb705e25f4be` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.529799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdba9ef-07e1-40b0-bf15-10f88c571d34 None admin] RBAC: Authorizing `identity:get_user(user_id=d3bdb62de62349b6b321bb705e25f4be)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:12.531383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdba9ef-07e1-40b0-bf15-10f88c571d34 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:12.533067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 63/117] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:12 2022] GET /identity/v3/users/d3bdb62de62349b6b321bb705e25f4be => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:12.538464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdba9ef-07e1-40b0-bf15-10f88c571d34 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.540521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b74d19f-e85e-44da-8b76-321bfa84fce7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.575289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52de44-81ef-4edf-9016-03857106616a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.576404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52de44-81ef-4edf-9016-03857106616a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.576932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52de44-81ef-4edf-9016-03857106616a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.577394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da52de44-81ef-4edf-9016-03857106616a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.589849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da52de44-81ef-4edf-9016-03857106616a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-da52de44-81ef-4edf-9016-03857106616a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:12.595335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:12.596435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:12.597368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 55/118] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:12.598950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52de44-81ef-4edf-9016-03857106616a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.611563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:12.648462 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdba9ef-07e1-40b0-bf15-10f88c571d34 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.655773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:12.682172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40656172-a3b4-4c48-817f-57593a515e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.682790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40656172-a3b4-4c48-817f-57593a515e4f None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.683413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40656172-a3b4-4c48-817f-57593a515e4f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.683684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40656172-a3b4-4c48-817f-57593a515e4f None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.684332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40656172-a3b4-4c48-817f-57593a515e4f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:12.685906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40656172-a3b4-4c48-817f-57593a515e4f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:12.699689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da52de44-81ef-4edf-9016-03857106616a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.713918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 64/119] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:47:12 2022] POST /identity/v3/projects => generated 302 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:47:12.715218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088e68f-02c6-4103-96df-3feda13070fd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.715471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088e68f-02c6-4103-96df-3feda13070fd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.715747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1088e68f-02c6-4103-96df-3feda13070fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.739289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90924a02-bdcd-4fc0-8b5a-d2bce4a73f62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.739932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90924a02-bdcd-4fc0-8b5a-d2bce4a73f62 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.740268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90924a02-bdcd-4fc0-8b5a-d2bce4a73f62 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.740549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90924a02-bdcd-4fc0-8b5a-d2bce4a73f62 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.741290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90924a02-bdcd-4fc0-8b5a-d2bce4a73f62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:12.741607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90924a02-bdcd-4fc0-8b5a-d2bce4a73f62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:12.742065 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1088e68f-02c6-4103-96df-3feda13070fd None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:12.743308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90924a02-bdcd-4fc0-8b5a-d2bce4a73f62 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:12.743414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1088e68f-02c6-4103-96df-3feda13070fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:12.755755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 56/120] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:12.759066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90924a02-bdcd-4fc0-8b5a-d2bce4a73f62 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.838111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 65/121] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:12.838111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088e68f-02c6-4103-96df-3feda13070fd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:12.851684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:12.895507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90924a02-bdcd-4fc0-8b5a-d2bce4a73f62 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.928955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:12.944839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628a47e-09a9-4d2c-a654-66a6944a5fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:12.945822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628a47e-09a9-4d2c-a654-66a6944a5fea None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:12.946863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628a47e-09a9-4d2c-a654-66a6944a5fea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:12.946863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628a47e-09a9-4d2c-a654-66a6944a5fea None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014/users/a0286a556e15432dbc2bd4ac3f2e801a/roles/c622170611454f18bf01aa658a8cc06e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:12.951289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1088e68f-02c6-4103-96df-3feda13070fd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:12.953302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628a47e-09a9-4d2c-a654-66a6944a5fea None admin] RBAC: Authorizing `identity:create_grant(project_id=ab8379e77b244774a8bf8329866e9014, user_id=a0286a556e15432dbc2bd4ac3f2e801a, role_id=c622170611454f18bf01aa658a8cc06e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:12.955709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628a47e-09a9-4d2c-a654-66a6944a5fea None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:12.972688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 57/122] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:47:12 2022] PUT /identity/v3/projects/ab8379e77b244774a8bf8329866e9014/users/a0286a556e15432dbc2bd4ac3f2e801a/roles/c622170611454f18bf01aa658a8cc06e => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:12.973440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628a47e-09a9-4d2c-a654-66a6944a5fea None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:13.044216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e8ee5-4c35-48e6-a0cb-28889e93c1b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:13.044970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e8ee5-4c35-48e6-a0cb-28889e93c1b5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:13.045445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e8ee5-4c35-48e6-a0cb-28889e93c1b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:13.045729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467e8ee5-4c35-48e6-a0cb-28889e93c1b5 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:13.050935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e8ee5-4c35-48e6-a0cb-28889e93c1b5 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:13.052458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467e8ee5-4c35-48e6-a0cb-28889e93c1b5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:13.054582 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 66/123] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:12 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:13.055376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e8ee5-4c35-48e6-a0cb-28889e93c1b5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:13.106715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628a47e-09a9-4d2c-a654-66a6944a5fea None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:13.147582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:13.155267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1949707-38d3-40cf-a3a2-9ebce2da133b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:13.156173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1949707-38d3-40cf-a3a2-9ebce2da133b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:13.156691 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1949707-38d3-40cf-a3a2-9ebce2da133b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:13.156972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1949707-38d3-40cf-a3a2-9ebce2da133b None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:13.157706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1949707-38d3-40cf-a3a2-9ebce2da133b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:13.158016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1949707-38d3-40cf-a3a2-9ebce2da133b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:13.159477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1949707-38d3-40cf-a3a2-9ebce2da133b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:13.177483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 58/124] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:13.177483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1949707-38d3-40cf-a3a2-9ebce2da133b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:13.240341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467e8ee5-4c35-48e6-a0cb-28889e93c1b5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:13.295056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa20b67-df76-4baf-ab8c-17c9f5e322d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:13.295726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa20b67-df76-4baf-ab8c-17c9f5e322d5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:13.296004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa20b67-df76-4baf-ab8c-17c9f5e322d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:13.296281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa20b67-df76-4baf-ab8c-17c9f5e322d5 None admin] PATH_INFO: `/v3/users/a0286a556e15432dbc2bd4ac3f2e801a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:13.297452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa20b67-df76-4baf-ab8c-17c9f5e322d5 None admin] RBAC: Authorizing `identity:get_user(user_id=a0286a556e15432dbc2bd4ac3f2e801a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:13.299509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa20b67-df76-4baf-ab8c-17c9f5e322d5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:13.302585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 67/125] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:13 2022] GET /identity/v3/users/a0286a556e15432dbc2bd4ac3f2e801a => generated 275 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:13.303090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa20b67-df76-4baf-ab8c-17c9f5e322d5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:13.323534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1949707-38d3-40cf-a3a2-9ebce2da133b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:13.330485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:13.367770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1d767-c674-4cd1-a177-35bb2dfb5b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:13.368608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1d767-c674-4cd1-a177-35bb2dfb5b35 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:13.368902 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1d767-c674-4cd1-a177-35bb2dfb5b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:13.369501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b1d767-c674-4cd1-a177-35bb2dfb5b35 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014/users/d3bdb62de62349b6b321bb705e25f4be/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:13.371962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1d767-c674-4cd1-a177-35bb2dfb5b35 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab8379e77b244774a8bf8329866e9014, user_id=d3bdb62de62349b6b321bb705e25f4be, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:13.373587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b1d767-c674-4cd1-a177-35bb2dfb5b35 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:13.391090 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 59/126] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:47:13 2022] PUT /identity/v3/projects/ab8379e77b244774a8bf8329866e9014/users/d3bdb62de62349b6b321bb705e25f4be/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:13.392004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1d767-c674-4cd1-a177-35bb2dfb5b35 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:13.424831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa20b67-df76-4baf-ab8c-17c9f5e322d5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:13.469182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:13.471917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce041f-96b1-4b0b-9006-d2568b1213a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:13.472896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce041f-96b1-4b0b-9006-d2568b1213a9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:13.473192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce041f-96b1-4b0b-9006-d2568b1213a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:13.473477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ce041f-96b1-4b0b-9006-d2568b1213a9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:13.487756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ce041f-96b1-4b0b-9006-d2568b1213a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b2ce041f-96b1-4b0b-9006-d2568b1213a9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:13.493984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:13.494924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:13.495706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 68/127] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:13.496888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce041f-96b1-4b0b-9006-d2568b1213a9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:13.544559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b1d767-c674-4cd1-a177-35bb2dfb5b35 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:13.604143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17554514-f83f-46dc-bcef-34ba69020622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:13.604987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17554514-f83f-46dc-bcef-34ba69020622 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:13.605227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17554514-f83f-46dc-bcef-34ba69020622 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:13.605561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17554514-f83f-46dc-bcef-34ba69020622 None admin] PATH_INFO: `/v3/users/8a0325a43d3343c98f2836ef6590b0d2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:13.625781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:13.635587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17554514-f83f-46dc-bcef-34ba69020622 None admin] RBAC: Authorizing `identity:get_user(user_id=8a0325a43d3343c98f2836ef6590b0d2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:13.637147 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17554514-f83f-46dc-bcef-34ba69020622 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:13.641588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 60/128] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:13 2022] GET /identity/v3/users/8a0325a43d3343c98f2836ef6590b0d2 => generated 289 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:13.646985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17554514-f83f-46dc-bcef-34ba69020622 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:13.663599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ce041f-96b1-4b0b-9006-d2568b1213a9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:13.710441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e669ea-b564-4ae9-bff8-c7013fe9e464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:13.711202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e669ea-b564-4ae9-bff8-c7013fe9e464 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:13.711558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e669ea-b564-4ae9-bff8-c7013fe9e464 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:13.711836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e669ea-b564-4ae9-bff8-c7013fe9e464 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:13.713091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e669ea-b564-4ae9-bff8-c7013fe9e464 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:13.714784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e669ea-b564-4ae9-bff8-c7013fe9e464 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:13.720014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 69/129] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:13 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:13.720014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e669ea-b564-4ae9-bff8-c7013fe9e464 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:13.805583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17554514-f83f-46dc-bcef-34ba69020622 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:13.822066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:13.846016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069366b-80de-42af-b305-6016af003fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:13.846848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069366b-80de-42af-b305-6016af003fda None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:13.847209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069366b-80de-42af-b305-6016af003fda None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:13.847577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6069366b-80de-42af-b305-6016af003fda None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:13.848404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069366b-80de-42af-b305-6016af003fda None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:13.848801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069366b-80de-42af-b305-6016af003fda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:13.850312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6069366b-80de-42af-b305-6016af003fda None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:13.858145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e669ea-b564-4ae9-bff8-c7013fe9e464 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:13.859778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 61/130] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:13.860521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069366b-80de-42af-b305-6016af003fda None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:13.900967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b0ddc-193f-4eef-a181-f0be3b8e318f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:13.901879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b0ddc-193f-4eef-a181-f0be3b8e318f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:13.902340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b0ddc-193f-4eef-a181-f0be3b8e318f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:13.902763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219b0ddc-193f-4eef-a181-f0be3b8e318f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:13.926188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:13.971360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6069366b-80de-42af-b305-6016af003fda None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:13.977518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b0ddc-193f-4eef-a181-f0be3b8e318f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:13.979296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-219b0ddc-193f-4eef-a181-f0be3b8e318f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:14.009308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bd80f-7a17-4b58-845a-2ef5963d432a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:14.010156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bd80f-7a17-4b58-845a-2ef5963d432a None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:14.010441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bd80f-7a17-4b58-845a-2ef5963d432a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:14.010715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142bd80f-7a17-4b58-845a-2ef5963d432a None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014/users/a0286a556e15432dbc2bd4ac3f2e801a/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:14.012672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bd80f-7a17-4b58-845a-2ef5963d432a None admin] RBAC: Authorizing `identity:create_grant(project_id=ab8379e77b244774a8bf8329866e9014, user_id=a0286a556e15432dbc2bd4ac3f2e801a, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:14.014583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142bd80f-7a17-4b58-845a-2ef5963d432a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:14.029205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 62/131] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:47:13 2022] PUT /identity/v3/projects/ab8379e77b244774a8bf8329866e9014/users/a0286a556e15432dbc2bd4ac3f2e801a/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:14.034160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bd80f-7a17-4b58-845a-2ef5963d432a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:14.069204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:14.126350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 70/132] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:14.132411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b0ddc-193f-4eef-a181-f0be3b8e318f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:14.146137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142bd80f-7a17-4b58-845a-2ef5963d432a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:14.228229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d36f6-5e03-4fc5-92e1-bbb39c536414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:14.228895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d36f6-5e03-4fc5-92e1-bbb39c536414 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:14.229175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d36f6-5e03-4fc5-92e1-bbb39c536414 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:14.229453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4d36f6-5e03-4fc5-92e1-bbb39c536414 None admin] PATH_INFO: `/v3/users/a0286a556e15432dbc2bd4ac3f2e801a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:14.233341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d36f6-5e03-4fc5-92e1-bbb39c536414 None admin] RBAC: Authorizing `identity:get_user(user_id=a0286a556e15432dbc2bd4ac3f2e801a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:14.234882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4d36f6-5e03-4fc5-92e1-bbb39c536414 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:14.237968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 63/133] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:14 2022] GET /identity/v3/users/a0286a556e15432dbc2bd4ac3f2e801a => generated 275 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:14.242457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d36f6-5e03-4fc5-92e1-bbb39c536414 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:14.245586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:14.344168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219b0ddc-193f-4eef-a181-f0be3b8e318f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:14.410296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30788a-7e00-4c88-9085-b6e7a24fab07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:14.411401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30788a-7e00-4c88-9085-b6e7a24fab07 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:14.411899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30788a-7e00-4c88-9085-b6e7a24fab07 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:14.412407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed30788a-7e00-4c88-9085-b6e7a24fab07 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:14.413796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30788a-7e00-4c88-9085-b6e7a24fab07 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:14.415521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed30788a-7e00-4c88-9085-b6e7a24fab07 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:14.417386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 71/134] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:14 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:14.423706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30788a-7e00-4c88-9085-b6e7a24fab07 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:14.425762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4d36f6-5e03-4fc5-92e1-bbb39c536414 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:14.427348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:14.465124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d063a3b-a9cb-447f-9ec5-9c22e8e9fd2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:14.465773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d063a3b-a9cb-447f-9ec5-9c22e8e9fd2b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:14.466057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d063a3b-a9cb-447f-9ec5-9c22e8e9fd2b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:14.466375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d063a3b-a9cb-447f-9ec5-9c22e8e9fd2b None admin] PATH_INFO: `/v3/projects/913b7cf334eb42248201177cfff59a6b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:14.467770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d063a3b-a9cb-447f-9ec5-9c22e8e9fd2b None admin] RBAC: Authorizing `identity:get_project(project_id=913b7cf334eb42248201177cfff59a6b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:14.469247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d063a3b-a9cb-447f-9ec5-9c22e8e9fd2b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:14.471284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 64/135] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:14 2022] GET /identity/v3/projects/913b7cf334eb42248201177cfff59a6b => generated 312 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:14.477076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d063a3b-a9cb-447f-9ec5-9c22e8e9fd2b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:14.536723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed30788a-7e00-4c88-9085-b6e7a24fab07 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:14.559449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:14.576889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86aab19-fd34-46ea-a300-94eb8afcbf6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:14.577931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aab19-fd34-46ea-a300-94eb8afcbf6e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:14.578636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aab19-fd34-46ea-a300-94eb8afcbf6e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:14.579156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86aab19-fd34-46ea-a300-94eb8afcbf6e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:14.580150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86aab19-fd34-46ea-a300-94eb8afcbf6e None admin] RBAC: Adding query filter params (role.id=15be693919a24ac69f0c9320bba5fac2, scope.project.id=ab8379e77b244774a8bf8329866e9014, user.id=8a0325a43d3343c98f2836ef6590b0d2) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:14.580669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86aab19-fd34-46ea-a300-94eb8afcbf6e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:14.582043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d063a3b-a9cb-447f-9ec5-9c22e8e9fd2b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:14.582583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86aab19-fd34-46ea-a300-94eb8afcbf6e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:14.589400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 72/136] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:14 2022] GET /identity/v3/role_assignments?user.id=8a0325a43d3343c98f2836ef6590b0d2&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=15be693919a24ac69f0c9320bba5fac2 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:14.622310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d00275-a3ef-4e0c-930b-fcfccfc1b0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:14.623142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d00275-a3ef-4e0c-930b-fcfccfc1b0aa None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:14.625083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d00275-a3ef-4e0c-930b-fcfccfc1b0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:14.625376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d00275-a3ef-4e0c-930b-fcfccfc1b0aa None admin] PATH_INFO: `/v3/projects/913b7cf334eb42248201177cfff59a6b/users/a0286a556e15432dbc2bd4ac3f2e801a/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:14.627540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d00275-a3ef-4e0c-930b-fcfccfc1b0aa None admin] RBAC: Authorizing `identity:create_grant(project_id=913b7cf334eb42248201177cfff59a6b, user_id=a0286a556e15432dbc2bd4ac3f2e801a, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:14.629031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d00275-a3ef-4e0c-930b-fcfccfc1b0aa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:14.656842 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 65/137] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:47:14 2022] PUT /identity/v3/projects/913b7cf334eb42248201177cfff59a6b/users/a0286a556e15432dbc2bd4ac3f2e801a/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 06:47:15.673627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850023d-2dcd-4ea0-84d1-df422e337d3c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:15.674209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850023d-2dcd-4ea0-84d1-df422e337d3c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:15.674209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5850023d-2dcd-4ea0-84d1-df422e337d3c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:15.675686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 73/138] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:15.685596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61266121-2cdc-4ccc-87da-04298ad5223a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:15.686526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61266121-2cdc-4ccc-87da-04298ad5223a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:15.686526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61266121-2cdc-4ccc-87da-04298ad5223a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:15.711422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-61266121-2cdc-4ccc-87da-04298ad5223a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:15.712648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-61266121-2cdc-4ccc-87da-04298ad5223a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:15.753716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:15.781784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef67d6-29af-4e1f-abd9-662639aed16f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:15.782124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef67d6-29af-4e1f-abd9-662639aed16f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:15.782368 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbef67d6-29af-4e1f-abd9-662639aed16f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:15.783870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 74/139] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:47:15.788821 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059161-15d7-40c6-8433-43935c45053d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:15.789111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059161-15d7-40c6-8433-43935c45053d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:15.789396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39059161-15d7-40c6-8433-43935c45053d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:15.811446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 66/140] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:15.827226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-39059161-15d7-40c6-8433-43935c45053d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:15.828470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-39059161-15d7-40c6-8433-43935c45053d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:15.911644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 75/141] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:16.253799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61266121-2cdc-4ccc-87da-04298ad5223a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:16.370620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61266121-2cdc-4ccc-87da-04298ad5223a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:16.413502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d9e76-1aa0-42f8-b418-c7482e28d4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:16.414243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d9e76-1aa0-42f8-b418-c7482e28d4b3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:16.414534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d9e76-1aa0-42f8-b418-c7482e28d4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:16.414807 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1d9e76-1aa0-42f8-b418-c7482e28d4b3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:16.415911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d9e76-1aa0-42f8-b418-c7482e28d4b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:16.417365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1d9e76-1aa0-42f8-b418-c7482e28d4b3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:16.418962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 67/142] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:16.424906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39059161-15d7-40c6-8433-43935c45053d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:16.445265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d9e76-1aa0-42f8-b418-c7482e28d4b3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:16.481472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:16.577854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39059161-15d7-40c6-8433-43935c45053d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:16.593952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:16.618511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551b177-0d7c-4d17-a3b9-fde3e3a4f701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:16.619397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551b177-0d7c-4d17-a3b9-fde3e3a4f701 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:16.619869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551b177-0d7c-4d17-a3b9-fde3e3a4f701 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:16.620336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f551b177-0d7c-4d17-a3b9-fde3e3a4f701 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:16.621326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551b177-0d7c-4d17-a3b9-fde3e3a4f701 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:16.623226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f551b177-0d7c-4d17-a3b9-fde3e3a4f701 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:16.648277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 76/143] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:47:16 2022] POST /identity/v3/users => generated 283 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:47:16.676540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1d9e76-1aa0-42f8-b418-c7482e28d4b3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:16.738967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a05b2a-fa92-4a4d-8b98-164bcea44325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:16.739656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a05b2a-fa92-4a4d-8b98-164bcea44325 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:16.739929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a05b2a-fa92-4a4d-8b98-164bcea44325 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:16.740214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a05b2a-fa92-4a4d-8b98-164bcea44325 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:16.757624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a05b2a-fa92-4a4d-8b98-164bcea44325 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-49a05b2a-fa92-4a4d-8b98-164bcea44325 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:16.763314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:16.764489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:16.765890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 68/144] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:16 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:16.770810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551b177-0d7c-4d17-a3b9-fde3e3a4f701 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:16.902632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f551b177-0d7c-4d17-a3b9-fde3e3a4f701 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:16.945236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd12d14-44be-4e80-8cb6-65f05bb69d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:16.946282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd12d14-44be-4e80-8cb6-65f05bb69d12 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:16.946786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd12d14-44be-4e80-8cb6-65f05bb69d12 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:16.947260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd12d14-44be-4e80-8cb6-65f05bb69d12 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:16.948141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd12d14-44be-4e80-8cb6-65f05bb69d12 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:16.948657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd12d14-44be-4e80-8cb6-65f05bb69d12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:16.950305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd12d14-44be-4e80-8cb6-65f05bb69d12 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:16.960287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 77/145] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:16 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:16.967215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a05b2a-fa92-4a4d-8b98-164bcea44325 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:17.131316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:17.143888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a05b2a-fa92-4a4d-8b98-164bcea44325 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:17.163999 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bc0b6-2cf5-452e-a2b7-1d619c08a9cb None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:17.164618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bc0b6-2cf5-452e-a2b7-1d619c08a9cb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:17.165153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bc0b6-2cf5-452e-a2b7-1d619c08a9cb None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:17.171497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 78/146] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:17.175685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49d827-62da-4fb4-b1be-e13a685b2acd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:17.176214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49d827-62da-4fb4-b1be-e13a685b2acd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:17.176691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e49d827-62da-4fb4-b1be-e13a685b2acd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:17.191521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8535a-2f65-48e5-b43c-378c937bb80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:17.192249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8535a-2f65-48e5-b43c-378c937bb80c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:17.192541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8535a-2f65-48e5-b43c-378c937bb80c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:17.192823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8535a-2f65-48e5-b43c-378c937bb80c None admin] PATH_INFO: `/v3/users/a0286a556e15432dbc2bd4ac3f2e801a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:17.194138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8535a-2f65-48e5-b43c-378c937bb80c None admin] RBAC: Authorizing `identity:get_user(user_id=a0286a556e15432dbc2bd4ac3f2e801a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:17.195628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8535a-2f65-48e5-b43c-378c937bb80c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:17.201807 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 69/147] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:16 2022] GET /identity/v3/users/a0286a556e15432dbc2bd4ac3f2e801a => generated 275 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:17.203225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8535a-2f65-48e5-b43c-378c937bb80c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:17.223923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9e49d827-62da-4fb4-b1be-e13a685b2acd None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:17.225616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9e49d827-62da-4fb4-b1be-e13a685b2acd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:17.278278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:17.347578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 79/148] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:17.387287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8535a-2f65-48e5-b43c-378c937bb80c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:17.442910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a94b51d-752c-474a-9da7-7eea21faeec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:17.443473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94b51d-752c-474a-9da7-7eea21faeec2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:17.443753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94b51d-752c-474a-9da7-7eea21faeec2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:17.444022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a94b51d-752c-474a-9da7-7eea21faeec2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:17.480968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:17.542874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94b51d-752c-474a-9da7-7eea21faeec2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:17.544404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a94b51d-752c-474a-9da7-7eea21faeec2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:17.710562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 70/149] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:17.716192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49d827-62da-4fb4-b1be-e13a685b2acd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:17.728163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935a9c-5028-4b6f-988d-c99c380b5f0a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:17.728450 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935a9c-5028-4b6f-988d-c99c380b5f0a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:17.729049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05935a9c-5028-4b6f-988d-c99c380b5f0a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:17.730308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 71/150] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:17.736250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250b6a2-a837-45eb-acd4-c60ae29142c1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:17.736250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250b6a2-a837-45eb-acd4-c60ae29142c1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:17.736583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250b6a2-a837-45eb-acd4-c60ae29142c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:17.760808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2250b6a2-a837-45eb-acd4-c60ae29142c1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:17.761985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2250b6a2-a837-45eb-acd4-c60ae29142c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:17.859388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:17.891841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 72/151] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:17.904645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e49d827-62da-4fb4-b1be-e13a685b2acd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:17.951593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554fa6e-138c-4bfa-9095-5f17ce545b78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:17.952282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554fa6e-138c-4bfa-9095-5f17ce545b78 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:17.952562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554fa6e-138c-4bfa-9095-5f17ce545b78 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:17.952837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2554fa6e-138c-4bfa-9095-5f17ce545b78 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:17.959042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554fa6e-138c-4bfa-9095-5f17ce545b78 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:17.960547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2554fa6e-138c-4bfa-9095-5f17ce545b78 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:17.962740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 80/152] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:17 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:17.971485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250b6a2-a837-45eb-acd4-c60ae29142c1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:18.040679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554fa6e-138c-4bfa-9095-5f17ce545b78 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:18.118462 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:18.162290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250b6a2-a837-45eb-acd4-c60ae29142c1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:18.237075 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98890c3d-a1fd-4111-b9c2-6ed8e74315f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:18.238044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98890c3d-a1fd-4111-b9c2-6ed8e74315f9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.238604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98890c3d-a1fd-4111-b9c2-6ed8e74315f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.239061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98890c3d-a1fd-4111-b9c2-6ed8e74315f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.239997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98890c3d-a1fd-4111-b9c2-6ed8e74315f9 None admin] RBAC: Adding query filter params (role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=ab8379e77b244774a8bf8329866e9014, user.id=a0286a556e15432dbc2bd4ac3f2e801a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:18.240490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98890c3d-a1fd-4111-b9c2-6ed8e74315f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:18.242197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98890c3d-a1fd-4111-b9c2-6ed8e74315f9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:18.254105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 73/153] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:17 2022] GET /identity/v3/role_assignments?user.id=a0286a556e15432dbc2bd4ac3f2e801a&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=c622170611454f18bf01aa658a8cc06e => generated 608 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:18.254105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f43cfc-64d9-4b9f-b077-820836a0bfb0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.254105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f43cfc-64d9-4b9f-b077-820836a0bfb0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.254105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f43cfc-64d9-4b9f-b077-820836a0bfb0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.254105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 74/154] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:18.254885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1677f-229c-4c74-9558-db334f1a5acf None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.255232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1677f-229c-4c74-9558-db334f1a5acf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.255590 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1677f-229c-4c74-9558-db334f1a5acf None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.256907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 75/155] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:18.257559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2554fa6e-138c-4bfa-9095-5f17ce545b78 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:18.262337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2377fa7-c858-4ded-b2fb-7088000a4d5d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.262649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2377fa7-c858-4ded-b2fb-7088000a4d5d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.262990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2377fa7-c858-4ded-b2fb-7088000a4d5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.285081 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b2377fa7-c858-4ded-b2fb-7088000a4d5d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:18.286483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b2377fa7-c858-4ded-b2fb-7088000a4d5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:18.331039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2207f-5048-4162-9574-11493254b905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:18.331701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2207f-5048-4162-9574-11493254b905 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.333013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2207f-5048-4162-9574-11493254b905 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.333292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2207f-5048-4162-9574-11493254b905 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.357508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2207f-5048-4162-9574-11493254b905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-69c2207f-5048-4162-9574-11493254b905 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:18.362952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:18.363819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:18.364581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:18.366454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:18.369329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 81/156] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:47:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:18.372124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df243508-554f-44a8-be61-78d4e2742aa3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.372412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df243508-554f-44a8-be61-78d4e2742aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.372716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df243508-554f-44a8-be61-78d4e2742aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.404308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 76/157] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:18.405399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2377fa7-c858-4ded-b2fb-7088000a4d5d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:18.413257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-df243508-554f-44a8-be61-78d4e2742aa3 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:18.414529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-df243508-554f-44a8-be61-78d4e2742aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:18.534890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 82/158] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:18.535816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df243508-554f-44a8-be61-78d4e2742aa3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:18.559258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:18.645249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df243508-554f-44a8-be61-78d4e2742aa3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:18.649708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2377fa7-c858-4ded-b2fb-7088000a4d5d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:18.682174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668381b2-331a-45c0-aa36-a5eaf7ceded6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:18.683675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668381b2-331a-45c0-aa36-a5eaf7ceded6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.684222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668381b2-331a-45c0-aa36-a5eaf7ceded6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.684704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668381b2-331a-45c0-aa36-a5eaf7ceded6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.704386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-668381b2-331a-45c0-aa36-a5eaf7ceded6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-668381b2-331a-45c0-aa36-a5eaf7ceded6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:18.710380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:18.711338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:18.712113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:18.718106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5f4ff-a08c-408f-b834-c353492c50ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:18.718106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5f4ff-a08c-408f-b834-c353492c50ed None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.718106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5f4ff-a08c-408f-b834-c353492c50ed None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.718106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d5f4ff-a08c-408f-b834-c353492c50ed None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.718426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5f4ff-a08c-408f-b834-c353492c50ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:18.718426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5f4ff-a08c-408f-b834-c353492c50ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:18.718832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d5f4ff-a08c-408f-b834-c353492c50ed None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:18.719495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 83/159] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:18.729679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668381b2-331a-45c0-aa36-a5eaf7ceded6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:18.732634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 77/160] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:18.739188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5f4ff-a08c-408f-b834-c353492c50ed None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:18.825481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:18.854465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:18.889873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668381b2-331a-45c0-aa36-a5eaf7ceded6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:18.905367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d5f4ff-a08c-408f-b834-c353492c50ed None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:18.945758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6712677-b06c-4a33-a883-79411849e849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:18.947161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6712677-b06c-4a33-a883-79411849e849 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.947624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6712677-b06c-4a33-a883-79411849e849 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.948078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6712677-b06c-4a33-a883-79411849e849 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.948938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6712677-b06c-4a33-a883-79411849e849 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:18.949435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6712677-b06c-4a33-a883-79411849e849 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:18.951374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6712677-b06c-4a33-a883-79411849e849 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:18.955079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6d8f2-cf78-4bf4-aa3b-54dbf919c080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:18.955079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6d8f2-cf78-4bf4-aa3b-54dbf919c080 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:18.955079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6d8f2-cf78-4bf4-aa3b-54dbf919c080 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:18.955079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6d8f2-cf78-4bf4-aa3b-54dbf919c080 None admin] PATH_INFO: `/v3/users/d3bdb62de62349b6b321bb705e25f4be` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:18.957205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6d8f2-cf78-4bf4-aa3b-54dbf919c080 None admin] RBAC: Authorizing `identity:get_user(user_id=d3bdb62de62349b6b321bb705e25f4be)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:18.961639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6d8f2-cf78-4bf4-aa3b-54dbf919c080 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:18.963555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 78/161] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:18 2022] GET /identity/v3/users/d3bdb62de62349b6b321bb705e25f4be => generated 247 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:18.970675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6d8f2-cf78-4bf4-aa3b-54dbf919c080 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:18.973452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 84/162] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:19.003251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6712677-b06c-4a33-a883-79411849e849 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:19.178039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6d8f2-cf78-4bf4-aa3b-54dbf919c080 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:19.192278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:19.219408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6712677-b06c-4a33-a883-79411849e849 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:19.239350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa3e27-4d20-4701-89eb-f45f903afe7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:19.239979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa3e27-4d20-4701-89eb-f45f903afe7f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:19.240257 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa3e27-4d20-4701-89eb-f45f903afe7f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:19.240527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0daa3e27-4d20-4701-89eb-f45f903afe7f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:19.281562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552163d-3985-47c2-a16e-a5ce43506ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:19.282231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552163d-3985-47c2-a16e-a5ce43506ad0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:19.282504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552163d-3985-47c2-a16e-a5ce43506ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:19.282768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7552163d-3985-47c2-a16e-a5ce43506ad0 None admin] PATH_INFO: `/v3/users/a0286a556e15432dbc2bd4ac3f2e801a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:19.284511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552163d-3985-47c2-a16e-a5ce43506ad0 None admin] RBAC: Authorizing `identity:get_user(user_id=a0286a556e15432dbc2bd4ac3f2e801a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:19.286050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7552163d-3985-47c2-a16e-a5ce43506ad0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:19.287683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 85/163] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:19 2022] GET /identity/v3/users/a0286a556e15432dbc2bd4ac3f2e801a => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:19.288447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552163d-3985-47c2-a16e-a5ce43506ad0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:19.317599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:19.369515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa3e27-4d20-4701-89eb-f45f903afe7f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:19.371034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0daa3e27-4d20-4701-89eb-f45f903afe7f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:19.495945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7552163d-3985-47c2-a16e-a5ce43506ad0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:19.542648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c932bda-9b60-4053-a44c-e9f52998e9e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:19.543288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c932bda-9b60-4053-a44c-e9f52998e9e8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:19.546147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c932bda-9b60-4053-a44c-e9f52998e9e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:19.546443 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c932bda-9b60-4053-a44c-e9f52998e9e8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:19.562690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c932bda-9b60-4053-a44c-e9f52998e9e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0c932bda-9b60-4053-a44c-e9f52998e9e8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:19.569141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:19.570007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:19.570811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 86/164] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:19.571703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c932bda-9b60-4053-a44c-e9f52998e9e8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:19.577911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 79/165] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 608 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:19.578787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa3e27-4d20-4701-89eb-f45f903afe7f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:19.596837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:19.703087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:19.747245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0daa3e27-4d20-4701-89eb-f45f903afe7f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:19.759012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c932bda-9b60-4053-a44c-e9f52998e9e8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:19.808687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fd290-4459-4978-958a-bc0507a6f4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:19.809308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fd290-4459-4978-958a-bc0507a6f4c4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:19.809582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fd290-4459-4978-958a-bc0507a6f4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:19.809851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fd290-4459-4978-958a-bc0507a6f4c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:19.816501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69007f-8648-4f73-8a38-37ce112243da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:19.817149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69007f-8648-4f73-8a38-37ce112243da None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:19.817422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69007f-8648-4f73-8a38-37ce112243da None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:19.817689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a69007f-8648-4f73-8a38-37ce112243da None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:19.840201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a69007f-8648-4f73-8a38-37ce112243da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1a69007f-8648-4f73-8a38-37ce112243da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:19.846602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:19.847469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:19.848256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 87/166] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:19.848975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69007f-8648-4f73-8a38-37ce112243da None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:19.915021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fd290-4459-4978-958a-bc0507a6f4c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:19.917089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fd290-4459-4978-958a-bc0507a6f4c4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:19.928914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:19.985908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a69007f-8648-4f73-8a38-37ce112243da None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:20.020219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:20.033374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e56e9b-b40e-42cd-b0a6-5885c79f6501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:20.034449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e56e9b-b40e-42cd-b0a6-5885c79f6501 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.034901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e56e9b-b40e-42cd-b0a6-5885c79f6501 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.035342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e56e9b-b40e-42cd-b0a6-5885c79f6501 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.036501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e56e9b-b40e-42cd-b0a6-5885c79f6501 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:20.037033 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e56e9b-b40e-42cd-b0a6-5885c79f6501 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:20.050112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e56e9b-b40e-42cd-b0a6-5885c79f6501 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:20.051620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 88/167] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:20.058770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e56e9b-b40e-42cd-b0a6-5885c79f6501 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:20.144659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 80/168] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 566 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:20.145537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fd290-4459-4978-958a-bc0507a6f4c4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:20.204018 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e56e9b-b40e-42cd-b0a6-5885c79f6501 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:20.242836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b333ba-a529-41b1-ae05-157786609e78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:20.243478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b333ba-a529-41b1-ae05-157786609e78 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.243753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b333ba-a529-41b1-ae05-157786609e78 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.244020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b333ba-a529-41b1-ae05-157786609e78 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.245836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b333ba-a529-41b1-ae05-157786609e78 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:20.247340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b333ba-a529-41b1-ae05-157786609e78 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:20.249290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 89/169] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:20 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:20.250048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b333ba-a529-41b1-ae05-157786609e78 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:20.340060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fd290-4459-4978-958a-bc0507a6f4c4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:20.358760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:20.375245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:20.405806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d4720-480a-4635-aedb-0959f096aef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:20.406508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d4720-480a-4635-aedb-0959f096aef2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.406792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d4720-480a-4635-aedb-0959f096aef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.407063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030d4720-480a-4635-aedb-0959f096aef2 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.407805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d4720-480a-4635-aedb-0959f096aef2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:20.408113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d4720-480a-4635-aedb-0959f096aef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:20.409529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030d4720-480a-4635-aedb-0959f096aef2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:20.429681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 81/170] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:20.430543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d4720-480a-4635-aedb-0959f096aef2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:20.450686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b333ba-a529-41b1-ae05-157786609e78 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:20.502773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa64707-05d0-4669-8ee5-74755d3d10af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:20.503417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa64707-05d0-4669-8ee5-74755d3d10af None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.503692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa64707-05d0-4669-8ee5-74755d3d10af None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.504057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa64707-05d0-4669-8ee5-74755d3d10af None admin] PATH_INFO: `/v3/users/8a0325a43d3343c98f2836ef6590b0d2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.505204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa64707-05d0-4669-8ee5-74755d3d10af None admin] RBAC: Authorizing `identity:get_user(user_id=8a0325a43d3343c98f2836ef6590b0d2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:20.506741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa64707-05d0-4669-8ee5-74755d3d10af None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:20.508565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 90/171] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:20 2022] GET /identity/v3/users/8a0325a43d3343c98f2836ef6590b0d2 => generated 289 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:20.509374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa64707-05d0-4669-8ee5-74755d3d10af None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:20.556339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030d4720-480a-4635-aedb-0959f096aef2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:20.604575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f508-f465-461e-af62-b797d708988f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:20.604575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f508-f465-461e-af62-b797d708988f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.604575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f508-f465-461e-af62-b797d708988f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.604575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd7f508-f465-461e-af62-b797d708988f None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.605000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7f508-f465-461e-af62-b797d708988f None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:20.605000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd7f508-f465-461e-af62-b797d708988f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:20.608924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 82/172] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:20 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:20.608924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f508-f465-461e-af62-b797d708988f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:20.641725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa64707-05d0-4669-8ee5-74755d3d10af None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:20.665057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:20.673479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:20.681890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d7d96-5aa4-4466-bdfa-565e189063b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:20.682689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d7d96-5aa4-4466-bdfa-565e189063b0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.683425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d7d96-5aa4-4466-bdfa-565e189063b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.683864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d7d96-5aa4-4466-bdfa-565e189063b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.684861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d7d96-5aa4-4466-bdfa-565e189063b0 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=ab8379e77b244774a8bf8329866e9014, user.id=d3bdb62de62349b6b321bb705e25f4be) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:20.685317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d7d96-5aa4-4466-bdfa-565e189063b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:20.687071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d7d96-5aa4-4466-bdfa-565e189063b0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:20.698245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 91/173] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:20 2022] GET /identity/v3/role_assignments?user.id=d3bdb62de62349b6b321bb705e25f4be&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:20.700257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c9289-9d58-4f01-90f8-9902d164315b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.700581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c9289-9d58-4f01-90f8-9902d164315b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.701005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865c9289-9d58-4f01-90f8-9902d164315b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.702533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 92/174] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:20.706590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c9289-9d58-4f01-90f8-9902d164315b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:20.744295 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd7f508-f465-461e-af62-b797d708988f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:20.793251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc24819-85e8-40c7-8f70-76f4e1565a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:20.793908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc24819-85e8-40c7-8f70-76f4e1565a0c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.794393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc24819-85e8-40c7-8f70-76f4e1565a0c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.795098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc24819-85e8-40c7-8f70-76f4e1565a0c None admin] PATH_INFO: `/v3/users/a0286a556e15432dbc2bd4ac3f2e801a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.796777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc24819-85e8-40c7-8f70-76f4e1565a0c None admin] RBAC: Authorizing `identity:get_user(user_id=a0286a556e15432dbc2bd4ac3f2e801a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:20.798298 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc24819-85e8-40c7-8f70-76f4e1565a0c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:20.800347 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 83/175] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:20 2022] GET /identity/v3/users/a0286a556e15432dbc2bd4ac3f2e801a => generated 275 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:20.801882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc24819-85e8-40c7-8f70-76f4e1565a0c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:20.848323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865c9289-9d58-4f01-90f8-9902d164315b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:20.901783 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d552706-c3e4-4512-b779-841905ddfb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:20.902882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d552706-c3e4-4512-b779-841905ddfb76 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.903158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d552706-c3e4-4512-b779-841905ddfb76 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.903429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d552706-c3e4-4512-b779-841905ddfb76 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.905142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d552706-c3e4-4512-b779-841905ddfb76 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:20.907378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d552706-c3e4-4512-b779-841905ddfb76 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:20.909732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 93/176] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:20 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:20.911515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb38041-1020-494e-a85f-248bc71d2a8f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.911795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb38041-1020-494e-a85f-248bc71d2a8f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.912369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb38041-1020-494e-a85f-248bc71d2a8f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.917815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 94/177] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:20 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:20.919292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226d68f-32d6-469c-8bf0-42d60e65eee9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.919643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226d68f-32d6-469c-8bf0-42d60e65eee9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.919918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9226d68f-32d6-469c-8bf0-42d60e65eee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.940797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9226d68f-32d6-469c-8bf0-42d60e65eee9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:20.942440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9226d68f-32d6-469c-8bf0-42d60e65eee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:20.947055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc24819-85e8-40c7-8f70-76f4e1565a0c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:20.962304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:20.992363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ed83e-e5e7-49e4-ab1c-62767a5fc2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:20.992986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ed83e-e5e7-49e4-ab1c-62767a5fc2ff None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:20.993266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ed83e-e5e7-49e4-ab1c-62767a5fc2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:20.993537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a56ed83e-e5e7-49e4-ab1c-62767a5fc2ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:20.994672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ed83e-e5e7-49e4-ab1c-62767a5fc2ff None admin] RBAC: Adding query filter params (role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=ab8379e77b244774a8bf8329866e9014, user.id=a0286a556e15432dbc2bd4ac3f2e801a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:20.994672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ed83e-e5e7-49e4-ab1c-62767a5fc2ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:20.996040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a56ed83e-e5e7-49e4-ab1c-62767a5fc2ff None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:21.004407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 84/178] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:20 2022] GET /identity/v3/role_assignments?user.id=a0286a556e15432dbc2bd4ac3f2e801a&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=2d59fcad66104464bbc848ef605f743e => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:21.005078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ed83e-e5e7-49e4-ab1c-62767a5fc2ff None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:21.018326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:21.034717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 95/179] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:21.035878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226d68f-32d6-469c-8bf0-42d60e65eee9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:21.141162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a56ed83e-e5e7-49e4-ab1c-62767a5fc2ff None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:21.179062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f0f26-5795-4233-8851-e4bd180ce34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:21.179624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0f26-5795-4233-8851-e4bd180ce34a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.179907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0f26-5795-4233-8851-e4bd180ce34a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.180183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f0f26-5795-4233-8851-e4bd180ce34a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.185909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9226d68f-32d6-469c-8bf0-42d60e65eee9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:21.227742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:21.244442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac2e941-52b7-47f3-b083-0b3205be1ca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:21.245132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2e941-52b7-47f3-b083-0b3205be1ca9 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.245413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2e941-52b7-47f3-b083-0b3205be1ca9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.245684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac2e941-52b7-47f3-b083-0b3205be1ca9 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014/users/8a0325a43d3343c98f2836ef6590b0d2/roles/15be693919a24ac69f0c9320bba5fac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.247707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2e941-52b7-47f3-b083-0b3205be1ca9 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab8379e77b244774a8bf8329866e9014, user_id=8a0325a43d3343c98f2836ef6590b0d2, role_id=15be693919a24ac69f0c9320bba5fac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:21.249238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac2e941-52b7-47f3-b083-0b3205be1ca9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:21.262201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f0f26-5795-4233-8851-e4bd180ce34a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:21.264042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f0f26-5795-4233-8851-e4bd180ce34a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:21.270106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 96/180] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:47:21 2022] PUT /identity/v3/projects/ab8379e77b244774a8bf8329866e9014/users/8a0325a43d3343c98f2836ef6590b0d2/roles/15be693919a24ac69f0c9320bba5fac2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:21.271780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd045ed-1eda-46ef-8074-5d8285683b6f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.272067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd045ed-1eda-46ef-8074-5d8285683b6f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.272338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd045ed-1eda-46ef-8074-5d8285683b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.297211 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9dd045ed-1eda-46ef-8074-5d8285683b6f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:21.298526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9dd045ed-1eda-46ef-8074-5d8285683b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:21.359883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:21.388128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 97/181] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:21.391356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4df524-ec4d-4df5-8fe7-4494f580740b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.391646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4df524-ec4d-4df5-8fe7-4494f580740b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.391924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4df524-ec4d-4df5-8fe7-4494f580740b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.397399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 98/182] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:21.397399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4df524-ec4d-4df5-8fe7-4494f580740b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:21.409402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 85/183] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:21.414479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd339c36-4471-4a05-bd69-766ccca08f83 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.414887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd339c36-4471-4a05-bd69-766ccca08f83 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.415170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd339c36-4471-4a05-bd69-766ccca08f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.437486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bd339c36-4471-4a05-bd69-766ccca08f83 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:21.438975 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bd339c36-4471-4a05-bd69-766ccca08f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:21.517246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4df524-ec4d-4df5-8fe7-4494f580740b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:21.524049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 86/184] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:21.530959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd339c36-4471-4a05-bd69-766ccca08f83 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:21.533723 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:21.559643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff641b-dacc-49ef-af86-64fb621567fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:21.560361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff641b-dacc-49ef-af86-64fb621567fa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.560640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff641b-dacc-49ef-af86-64fb621567fa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.560914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbff641b-dacc-49ef-af86-64fb621567fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.572484 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbff641b-dacc-49ef-af86-64fb621567fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-dbff641b-dacc-49ef-af86-64fb621567fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:21.578440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:21.579328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:21.580114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 99/185] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:47:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:21.590895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff641b-dacc-49ef-af86-64fb621567fa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:21.650091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:21.726113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbff641b-dacc-49ef-af86-64fb621567fa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:21.749526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd339c36-4471-4a05-bd69-766ccca08f83 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:21.768670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979ccea-f340-42ce-859c-b8764954de86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:21.769507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979ccea-f340-42ce-859c-b8764954de86 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.769917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979ccea-f340-42ce-859c-b8764954de86 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.770373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c979ccea-f340-42ce-859c-b8764954de86 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.771259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979ccea-f340-42ce-859c-b8764954de86 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:21.771689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979ccea-f340-42ce-859c-b8764954de86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:21.773227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c979ccea-f340-42ce-859c-b8764954de86 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:21.782448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 100/186] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:21.783432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979ccea-f340-42ce-859c-b8764954de86 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:21.785593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f151f5-fb8b-4e69-958b-ac13083ffb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:21.786411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f151f5-fb8b-4e69-958b-ac13083ffb48 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.786829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f151f5-fb8b-4e69-958b-ac13083ffb48 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.787331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f151f5-fb8b-4e69-958b-ac13083ffb48 None admin] PATH_INFO: `/v3/projects/913b7cf334eb42248201177cfff59a6b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.788556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f151f5-fb8b-4e69-958b-ac13083ffb48 None admin] RBAC: Authorizing `identity:get_project(project_id=913b7cf334eb42248201177cfff59a6b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:21.790034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f151f5-fb8b-4e69-958b-ac13083ffb48 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:21.791707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 87/187] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:21 2022] GET /identity/v3/projects/913b7cf334eb42248201177cfff59a6b => generated 312 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:21.792458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f151f5-fb8b-4e69-958b-ac13083ffb48 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:21.867629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:21.893670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c979ccea-f340-42ce-859c-b8764954de86 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:21.895577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:21.903005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f151f5-fb8b-4e69-958b-ac13083ffb48 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:21.932425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bbfad-0103-4410-b317-4836c6fdc929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:21.933841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bbfad-0103-4410-b317-4836c6fdc929 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.934312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bbfad-0103-4410-b317-4836c6fdc929 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.934724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bbfad-0103-4410-b317-4836c6fdc929 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.937744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21738b16-1130-4d2a-9a99-85a2e901e479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:21.938418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21738b16-1130-4d2a-9a99-85a2e901e479 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:21.938699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21738b16-1130-4d2a-9a99-85a2e901e479 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:21.938973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21738b16-1130-4d2a-9a99-85a2e901e479 None admin] PATH_INFO: `/v3/users/4b1402b6b15e4995ab5c7a6789531d1a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:21.946165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9bbfad-0103-4410-b317-4836c6fdc929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5e9bbfad-0103-4410-b317-4836c6fdc929 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:21.951992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:21.953019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:21.953964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:21.954705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21738b16-1130-4d2a-9a99-85a2e901e479 None admin] RBAC: Authorizing `identity:get_user(user_id=4b1402b6b15e4995ab5c7a6789531d1a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:21.954705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21738b16-1130-4d2a-9a99-85a2e901e479 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:21.955448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 88/188] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:21 2022] GET /identity/v3/users/4b1402b6b15e4995ab5c7a6789531d1a => generated 283 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:21.956208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21738b16-1130-4d2a-9a99-85a2e901e479 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:21.956736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 101/189] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:21.957631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bbfad-0103-4410-b317-4836c6fdc929 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:22.066385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bbfad-0103-4410-b317-4836c6fdc929 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:22.068289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21738b16-1130-4d2a-9a99-85a2e901e479 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:22.097642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:22.102983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c18d6-7b07-44d8-a103-037af9d8bc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:22.103619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c18d6-7b07-44d8-a103-037af9d8bc2c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:22.103909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c18d6-7b07-44d8-a103-037af9d8bc2c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:22.104179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31c18d6-7b07-44d8-a103-037af9d8bc2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:22.105273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c18d6-7b07-44d8-a103-037af9d8bc2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:22.106654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891d11a-4a5d-406a-ac8e-e9b50a353858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:22.106769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31c18d6-7b07-44d8-a103-037af9d8bc2c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:22.108041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891d11a-4a5d-406a-ac8e-e9b50a353858 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:22.108330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891d11a-4a5d-406a-ac8e-e9b50a353858 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:22.110332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3891d11a-4a5d-406a-ac8e-e9b50a353858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:22.110426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 102/190] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:22.110500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891d11a-4a5d-406a-ac8e-e9b50a353858 None admin] RBAC: Adding query filter params (role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=913b7cf334eb42248201177cfff59a6b, user.id=a0286a556e15432dbc2bd4ac3f2e801a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:22.110590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c18d6-7b07-44d8-a103-037af9d8bc2c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:22.110662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891d11a-4a5d-406a-ac8e-e9b50a353858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:22.111084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3891d11a-4a5d-406a-ac8e-e9b50a353858 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:22.118716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 89/191] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:21 2022] GET /identity/v3/role_assignments?user.id=a0286a556e15432dbc2bd4ac3f2e801a&scope.project.id=913b7cf334eb42248201177cfff59a6b&role.id=2d59fcad66104464bbc848ef605f743e => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:22.119524 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891d11a-4a5d-406a-ac8e-e9b50a353858 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:22.154625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:22.244832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31c18d6-7b07-44d8-a103-037af9d8bc2c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:22.257272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3891d11a-4a5d-406a-ac8e-e9b50a353858 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:22.289461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b0dae-394b-4563-930b-47f03f1bbfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:22.290147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b0dae-394b-4563-930b-47f03f1bbfe4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:22.290978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b0dae-394b-4563-930b-47f03f1bbfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:22.291270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9b0dae-394b-4563-930b-47f03f1bbfe4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:22.306657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52854745-b5ed-49e8-b3e5-3c8bfc54e995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:22.308550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52854745-b5ed-49e8-b3e5-3c8bfc54e995 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:22.309063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52854745-b5ed-49e8-b3e5-3c8bfc54e995 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:22.309561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52854745-b5ed-49e8-b3e5-3c8bfc54e995 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:22.310508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52854745-b5ed-49e8-b3e5-3c8bfc54e995 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:22.311024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52854745-b5ed-49e8-b3e5-3c8bfc54e995 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:22.312644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52854745-b5ed-49e8-b3e5-3c8bfc54e995 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:22.323892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 90/192] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:22.324478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52854745-b5ed-49e8-b3e5-3c8bfc54e995 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:22.376496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b0dae-394b-4563-930b-47f03f1bbfe4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:22.378000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f9b0dae-394b-4563-930b-47f03f1bbfe4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:22.401417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:22.415796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:22.461479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52854745-b5ed-49e8-b3e5-3c8bfc54e995 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:22.486857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 103/193] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:22.487710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b0dae-394b-4563-930b-47f03f1bbfe4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:22.498506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50469d0-6ab3-48c1-88ea-4ca063074d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:22.499129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50469d0-6ab3-48c1-88ea-4ca063074d84 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:22.499413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50469d0-6ab3-48c1-88ea-4ca063074d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:22.499689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50469d0-6ab3-48c1-88ea-4ca063074d84 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:22.500376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50469d0-6ab3-48c1-88ea-4ca063074d84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:22.501972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50469d0-6ab3-48c1-88ea-4ca063074d84 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:22.527500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 91/194] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:47:22 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:47:22.528262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50469d0-6ab3-48c1-88ea-4ca063074d84 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:22.616327 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9b0dae-394b-4563-930b-47f03f1bbfe4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:22.652232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:22.658973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8772ab4-315d-48c4-be25-fd08574078bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:22.659851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8772ab4-315d-48c4-be25-fd08574078bf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:22.660199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8772ab4-315d-48c4-be25-fd08574078bf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:22.660628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8772ab4-315d-48c4-be25-fd08574078bf None admin] PATH_INFO: `/v3/users/4b1402b6b15e4995ab5c7a6789531d1a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:22.661941 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8772ab4-315d-48c4-be25-fd08574078bf None admin] RBAC: Authorizing `identity:get_user(user_id=4b1402b6b15e4995ab5c7a6789531d1a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:22.663639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8772ab4-315d-48c4-be25-fd08574078bf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:22.665513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 104/195] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:22 2022] GET /identity/v3/users/4b1402b6b15e4995ab5c7a6789531d1a => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:22.671553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8772ab4-315d-48c4-be25-fd08574078bf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:22.683512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50469d0-6ab3-48c1-88ea-4ca063074d84 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:22.699247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:22.738365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1a971-926a-4704-a902-10faba34f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:22.739163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1a971-926a-4704-a902-10faba34f530 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:22.739446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1a971-926a-4704-a902-10faba34f530 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:22.739847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da1a971-926a-4704-a902-10faba34f530 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:22.741271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1a971-926a-4704-a902-10faba34f530 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:22.743614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da1a971-926a-4704-a902-10faba34f530 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:22.751119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 92/196] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:22 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:22.753168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1a971-926a-4704-a902-10faba34f530 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:22.792381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8772ab4-315d-48c4-be25-fd08574078bf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:22.838582 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d27cd-61b2-4273-82d7-061a04abdd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:22.839504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d27cd-61b2-4273-82d7-061a04abdd5d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:22.840288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d27cd-61b2-4273-82d7-061a04abdd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:22.840761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d27cd-61b2-4273-82d7-061a04abdd5d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:22.867188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da1a971-926a-4704-a902-10faba34f530 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:22.911387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94844b7-a00c-44d7-bcb7-48b5c8208580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:22.912042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94844b7-a00c-44d7-bcb7-48b5c8208580 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:22.912436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94844b7-a00c-44d7-bcb7-48b5c8208580 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:22.912721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94844b7-a00c-44d7-bcb7-48b5c8208580 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:22.913469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94844b7-a00c-44d7-bcb7-48b5c8208580 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=fe3324999301468c8543ebc979dbd819, user.id=4b1402b6b15e4995ab5c7a6789531d1a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:22.913781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94844b7-a00c-44d7-bcb7-48b5c8208580 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:22.915308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94844b7-a00c-44d7-bcb7-48b5c8208580 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:22.920849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:22.922219 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 93/197] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:22 2022] GET /identity/v3/role_assignments?user.id=4b1402b6b15e4995ab5c7a6789531d1a&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:22.924032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d27cd-61b2-4273-82d7-061a04abdd5d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:22.925651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d27cd-61b2-4273-82d7-061a04abdd5d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:23.039942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 105/198] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:23.046211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94844b7-a00c-44d7-bcb7-48b5c8208580 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:23.099212 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:23.160534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94844b7-a00c-44d7-bcb7-48b5c8208580 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:23.199163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe5597-fd9a-40a6-9e80-96fcaf3ade80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:23.200146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe5597-fd9a-40a6-9e80-96fcaf3ade80 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:23.200599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe5597-fd9a-40a6-9e80-96fcaf3ade80 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:23.201095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe5597-fd9a-40a6-9e80-96fcaf3ade80 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:23.202701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe5597-fd9a-40a6-9e80-96fcaf3ade80 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:23.204432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fe5597-fd9a-40a6-9e80-96fcaf3ade80 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:23.206393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 94/199] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:23 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:23.212506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d27cd-61b2-4273-82d7-061a04abdd5d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:23.333251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:23.336833 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d27cd-61b2-4273-82d7-061a04abdd5d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:23.393124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b107cf5e-fa32-442f-912c-671a187beaaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:23.393738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107cf5e-fa32-442f-912c-671a187beaaf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:23.394012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107cf5e-fa32-442f-912c-671a187beaaf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:23.394313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b107cf5e-fa32-442f-912c-671a187beaaf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:23.395040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107cf5e-fa32-442f-912c-671a187beaaf None admin] RBAC: Adding query filter params (role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=fe3324999301468c8543ebc979dbd819, user.id=4b1402b6b15e4995ab5c7a6789531d1a) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:23.395344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107cf5e-fa32-442f-912c-671a187beaaf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:23.396799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b107cf5e-fa32-442f-912c-671a187beaaf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:23.402445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 106/200] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:23 2022] GET /identity/v3/role_assignments?user.id=4b1402b6b15e4995ab5c7a6789531d1a&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=c622170611454f18bf01aa658a8cc06e => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:23.711305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b34cf1-7f88-4cad-b883-efdc9546fbc8 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:23.711779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b34cf1-7f88-4cad-b883-efdc9546fbc8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:23.712170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b34cf1-7f88-4cad-b883-efdc9546fbc8 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:23.713918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 95/201] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:23.720100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35274fc3-c173-4326-b9ad-c3790c10030a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:23.720379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35274fc3-c173-4326-b9ad-c3790c10030a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:23.720650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35274fc3-c173-4326-b9ad-c3790c10030a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:23.751300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-35274fc3-c173-4326-b9ad-c3790c10030a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:23.752645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-35274fc3-c173-4326-b9ad-c3790c10030a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:23.823529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 107/202] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:24.096712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b34cf1-7f88-4cad-b883-efdc9546fbc8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:24.238016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b34cf1-7f88-4cad-b883-efdc9546fbc8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:24.249047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:24.272987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb1836-6791-4909-b711-9d274bd1f50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:24.273922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb1836-6791-4909-b711-9d274bd1f50b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:24.274224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb1836-6791-4909-b711-9d274bd1f50b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:24.274494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb1836-6791-4909-b711-9d274bd1f50b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:24.284846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb1836-6791-4909-b711-9d274bd1f50b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2aeb1836-6791-4909-b711-9d274bd1f50b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:24.289675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:24.290586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:24.291389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 96/203] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:24.296977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35274fc3-c173-4326-b9ad-c3790c10030a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:24.421642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:24.430391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35274fc3-c173-4326-b9ad-c3790c10030a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:24.466244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d9c02-eaa3-475d-b944-495ab797caaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:24.467009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d9c02-eaa3-475d-b944-495ab797caaf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:24.467345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d9c02-eaa3-475d-b944-495ab797caaf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:24.467621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13d9c02-eaa3-475d-b944-495ab797caaf None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:24.468296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d9c02-eaa3-475d-b944-495ab797caaf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:24.468689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d9c02-eaa3-475d-b944-495ab797caaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:24.470296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13d9c02-eaa3-475d-b944-495ab797caaf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:24.479612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 108/204] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:24.485323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb1836-6791-4909-b711-9d274bd1f50b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:24.646552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb1836-6791-4909-b711-9d274bd1f50b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:24.704333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5aff6-547b-4feb-b648-86cbb2cdf6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:24.705364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5aff6-547b-4feb-b648-86cbb2cdf6d3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:24.705888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5aff6-547b-4feb-b648-86cbb2cdf6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:24.706398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a5aff6-547b-4feb-b648-86cbb2cdf6d3 None admin] PATH_INFO: `/v3/users/8a0325a43d3343c98f2836ef6590b0d2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:24.710818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5aff6-547b-4feb-b648-86cbb2cdf6d3 None admin] RBAC: Authorizing `identity:get_user(user_id=8a0325a43d3343c98f2836ef6590b0d2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:24.712543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a5aff6-547b-4feb-b648-86cbb2cdf6d3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:24.714725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 97/205] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:24 2022] GET /identity/v3/users/8a0325a43d3343c98f2836ef6590b0d2 => generated 289 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:24.720490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d9c02-eaa3-475d-b944-495ab797caaf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:24.874047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13d9c02-eaa3-475d-b944-495ab797caaf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:24.912033 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cc10a-9d55-4559-b220-0c7c91f8075d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:24.912654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cc10a-9d55-4559-b220-0c7c91f8075d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:24.912929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cc10a-9d55-4559-b220-0c7c91f8075d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:24.913203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7cc10a-9d55-4559-b220-0c7c91f8075d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:24.945891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:25.001625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cc10a-9d55-4559-b220-0c7c91f8075d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:25.003463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7cc10a-9d55-4559-b220-0c7c91f8075d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:25.130486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 109/206] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:25.135928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5aff6-547b-4feb-b648-86cbb2cdf6d3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:25.194710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:25.261983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a5aff6-547b-4feb-b648-86cbb2cdf6d3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:25.297298 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9670dee-c570-45a9-8c9a-aaef96589b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:25.297945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9670dee-c570-45a9-8c9a-aaef96589b52 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:25.298251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9670dee-c570-45a9-8c9a-aaef96589b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:25.298518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9670dee-c570-45a9-8c9a-aaef96589b52 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:25.299652 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9670dee-c570-45a9-8c9a-aaef96589b52 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:25.301106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9670dee-c570-45a9-8c9a-aaef96589b52 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:25.302767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 98/207] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:25 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:25.309088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cc10a-9d55-4559-b220-0c7c91f8075d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:25.372817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:25.405698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bfb12-93da-4fca-882d-d0dd8abeb9ec None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:25.406048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bfb12-93da-4fca-882d-d0dd8abeb9ec None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:25.406316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332bfb12-93da-4fca-882d-d0dd8abeb9ec None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:25.407732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 99/208] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:25.415210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ed8e8-132c-4c46-a7de-dfa5f85be3d4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:25.415487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ed8e8-132c-4c46-a7de-dfa5f85be3d4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:25.415754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2ed8e8-132c-4c46-a7de-dfa5f85be3d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:25.436252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ea2ed8e8-132c-4c46-a7de-dfa5f85be3d4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:25.438226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ea2ed8e8-132c-4c46-a7de-dfa5f85be3d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:25.469183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7cc10a-9d55-4559-b220-0c7c91f8075d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:25.522934 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 100/209] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:25.529740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911d4baf-0649-4900-b182-86f6d511c217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:25.530638 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d4baf-0649-4900-b182-86f6d511c217 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:25.531118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d4baf-0649-4900-b182-86f6d511c217 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:25.531590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911d4baf-0649-4900-b182-86f6d511c217 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:25.532537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d4baf-0649-4900-b182-86f6d511c217 None admin] RBAC: Adding query filter params (role.id=15be693919a24ac69f0c9320bba5fac2, scope.project.id=ab8379e77b244774a8bf8329866e9014, user.id=8a0325a43d3343c98f2836ef6590b0d2) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:25.533045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d4baf-0649-4900-b182-86f6d511c217 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:25.534878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911d4baf-0649-4900-b182-86f6d511c217 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:25.541261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 110/210] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:25 2022] GET /identity/v3/role_assignments?user.id=8a0325a43d3343c98f2836ef6590b0d2&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=15be693919a24ac69f0c9320bba5fac2 => generated 608 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:25.810144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041aa008-e668-4b43-a30f-782b45de6ea2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:25.810504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041aa008-e668-4b43-a30f-782b45de6ea2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:25.810736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041aa008-e668-4b43-a30f-782b45de6ea2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:25.812395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 101/211] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:25.817840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87389b8-bb3a-4074-a874-b9740b696bfd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:25.818441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87389b8-bb3a-4074-a874-b9740b696bfd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:25.818915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87389b8-bb3a-4074-a874-b9740b696bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:25.842625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a87389b8-bb3a-4074-a874-b9740b696bfd None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:25.844175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a87389b8-bb3a-4074-a874-b9740b696bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:25.849643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7be3a2-8a58-4f31-b858-83992ebb6314 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:25.849931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7be3a2-8a58-4f31-b858-83992ebb6314 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:25.850258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7be3a2-8a58-4f31-b858-83992ebb6314 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:25.851608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 102/212] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:47:25.858237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbece1-c6a5-472c-9cff-8f5e4fd06e32 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:25.858519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbece1-c6a5-472c-9cff-8f5e4fd06e32 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:25.858797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cbece1-c6a5-472c-9cff-8f5e4fd06e32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:25.884102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-17cbece1-c6a5-472c-9cff-8f5e4fd06e32 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:25.885405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-17cbece1-c6a5-472c-9cff-8f5e4fd06e32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:25.920363 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 111/213] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:25.921016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:25.921532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87389b8-bb3a-4074-a874-b9740b696bfd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:25.964062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 103/214] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:25.983381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:26.036269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87389b8-bb3a-4074-a874-b9740b696bfd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.079981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639803b8-13b5-4d74-bf6d-4b92a57e2dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.080559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639803b8-13b5-4d74-bf6d-4b92a57e2dcd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.080835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639803b8-13b5-4d74-bf6d-4b92a57e2dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.081105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639803b8-13b5-4d74-bf6d-4b92a57e2dcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.093692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639803b8-13b5-4d74-bf6d-4b92a57e2dcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-639803b8-13b5-4d74-bf6d-4b92a57e2dcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:26.098765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:26.099657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:26.100468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 112/215] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:26.105977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbece1-c6a5-472c-9cff-8f5e4fd06e32 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:26.179142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363545e-6e58-4f35-8707-48191a2cc017 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.179669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363545e-6e58-4f35-8707-48191a2cc017 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.179954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a363545e-6e58-4f35-8707-48191a2cc017 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.182372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 113/216] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:47:26.187259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb5e6d-dafe-41af-825d-9fa6d3edd559 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.187658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb5e6d-dafe-41af-825d-9fa6d3edd559 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.188038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3eb5e6d-dafe-41af-825d-9fa6d3edd559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.212669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f3eb5e6d-dafe-41af-825d-9fa6d3edd559 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:26.214620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f3eb5e6d-dafe-41af-825d-9fa6d3edd559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:26.217144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cbece1-c6a5-472c-9cff-8f5e4fd06e32 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.257730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e45fe-c8a2-440c-96d0-f8dbd50b3aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.258384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e45fe-c8a2-440c-96d0-f8dbd50b3aac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.258664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e45fe-c8a2-440c-96d0-f8dbd50b3aac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.258934 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e45fe-c8a2-440c-96d0-f8dbd50b3aac None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.259711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e45fe-c8a2-440c-96d0-f8dbd50b3aac None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:26.260043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e45fe-c8a2-440c-96d0-f8dbd50b3aac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:26.261491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e45fe-c8a2-440c-96d0-f8dbd50b3aac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:26.270307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 104/217] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:26.271126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e45fe-c8a2-440c-96d0-f8dbd50b3aac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:26.287885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 114/218] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:26.288763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb5e6d-dafe-41af-825d-9fa6d3edd559 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:26.302020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:26.332981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:26.385343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e45fe-c8a2-440c-96d0-f8dbd50b3aac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.400173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3eb5e6d-dafe-41af-825d-9fa6d3edd559 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.420822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864d3ed-237c-4914-afa5-5722164b518f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.421619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864d3ed-237c-4914-afa5-5722164b518f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.421897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864d3ed-237c-4914-afa5-5722164b518f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.422207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4864d3ed-237c-4914-afa5-5722164b518f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.423331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864d3ed-237c-4914-afa5-5722164b518f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:26.424805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4864d3ed-237c-4914-afa5-5722164b518f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:26.426446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 105/219] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:26.431652 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864d3ed-237c-4914-afa5-5722164b518f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:26.434799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17b661-2f7f-4c17-9bc9-c388d6a1503c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.436538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17b661-2f7f-4c17-9bc9-c388d6a1503c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.436829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17b661-2f7f-4c17-9bc9-c388d6a1503c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.437107 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba17b661-2f7f-4c17-9bc9-c388d6a1503c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.448715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba17b661-2f7f-4c17-9bc9-c388d6a1503c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ba17b661-2f7f-4c17-9bc9-c388d6a1503c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:26.454044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:26.454935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:26.455725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 115/220] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:47:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:26.456373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17b661-2f7f-4c17-9bc9-c388d6a1503c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:26.531813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:26.538938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4864d3ed-237c-4914-afa5-5722164b518f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.561392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:26.565356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba17b661-2f7f-4c17-9bc9-c388d6a1503c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.574179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837eeb8-1844-4b26-ad8e-79b38c04dbb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.574826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837eeb8-1844-4b26-ad8e-79b38c04dbb3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.575110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837eeb8-1844-4b26-ad8e-79b38c04dbb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.575401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7837eeb8-1844-4b26-ad8e-79b38c04dbb3 None admin] PATH_INFO: `/v3/users/3fbe94213ed84f34a7529106f60d03b8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.588789 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837eeb8-1844-4b26-ad8e-79b38c04dbb3 None admin] RBAC: Authorizing `identity:get_user(user_id=3fbe94213ed84f34a7529106f60d03b8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:26.590220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7837eeb8-1844-4b26-ad8e-79b38c04dbb3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:26.591886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 106/221] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:26 2022] GET /identity/v3/users/3fbe94213ed84f34a7529106f60d03b8 => generated 297 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:26.596733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837eeb8-1844-4b26-ad8e-79b38c04dbb3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:26.603216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1c891-9cb7-4d50-98a8-7a4715b48524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.603836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1c891-9cb7-4d50-98a8-7a4715b48524 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.604109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1c891-9cb7-4d50-98a8-7a4715b48524 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.604375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a1c891-9cb7-4d50-98a8-7a4715b48524 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.605047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1c891-9cb7-4d50-98a8-7a4715b48524 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:26.606585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a1c891-9cb7-4d50-98a8-7a4715b48524 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:26.629468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 116/222] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:47:26 2022] POST /identity/v3/users => generated 297 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:47:26.630317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1c891-9cb7-4d50-98a8-7a4715b48524 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:26.712426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7837eeb8-1844-4b26-ad8e-79b38c04dbb3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.748963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dfa00-1316-4ef8-b401-8aa0013dc05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.749446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a1c891-9cb7-4d50-98a8-7a4715b48524 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.749617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dfa00-1316-4ef8-b401-8aa0013dc05d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.749861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dfa00-1316-4ef8-b401-8aa0013dc05d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.750156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743dfa00-1316-4ef8-b401-8aa0013dc05d None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.750830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dfa00-1316-4ef8-b401-8aa0013dc05d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:26.751135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dfa00-1316-4ef8-b401-8aa0013dc05d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:26.752546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743dfa00-1316-4ef8-b401-8aa0013dc05d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:26.761450 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 107/223] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:26.762367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dfa00-1316-4ef8-b401-8aa0013dc05d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:26.782764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:26.792016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71da7c-545d-4760-b342-2752016ecad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.792679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71da7c-545d-4760-b342-2752016ecad8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.792962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71da7c-545d-4760-b342-2752016ecad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.793234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f71da7c-545d-4760-b342-2752016ecad8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.804435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f71da7c-545d-4760-b342-2752016ecad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6f71da7c-545d-4760-b342-2752016ecad8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:26.811026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:26.812061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 117/224] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:26.815106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71da7c-545d-4760-b342-2752016ecad8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:26.833855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:26.890102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743dfa00-1316-4ef8-b401-8aa0013dc05d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.941232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed8a3a-c685-4956-9fcd-5a38a4f17ed2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.941920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed8a3a-c685-4956-9fcd-5a38a4f17ed2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.942175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed8a3a-c685-4956-9fcd-5a38a4f17ed2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.942449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed8a3a-c685-4956-9fcd-5a38a4f17ed2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.943555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f71da7c-545d-4760-b342-2752016ecad8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:26.980438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb151ab-edb4-4ce2-9055-f3c9464b9cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:26.981081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb151ab-edb4-4ce2-9055-f3c9464b9cb7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:26.981354 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb151ab-edb4-4ce2-9055-f3c9464b9cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:26.981621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb151ab-edb4-4ce2-9055-f3c9464b9cb7 None admin] PATH_INFO: `/v3/users/4b1402b6b15e4995ab5c7a6789531d1a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:26.983168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb151ab-edb4-4ce2-9055-f3c9464b9cb7 None admin] RBAC: Authorizing `identity:get_user(user_id=4b1402b6b15e4995ab5c7a6789531d1a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:26.984633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb151ab-edb4-4ce2-9055-f3c9464b9cb7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:26.986240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 118/225] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:26 2022] GET /identity/v3/users/4b1402b6b15e4995ab5c7a6789531d1a => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:26.987004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb151ab-edb4-4ce2-9055-f3c9464b9cb7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:27.027884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed8a3a-c685-4956-9fcd-5a38a4f17ed2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:27.029989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ed8a3a-c685-4956-9fcd-5a38a4f17ed2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:27.048755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:27.081992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:27.102571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb151ab-edb4-4ce2-9055-f3c9464b9cb7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:27.139855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e33d7-a6dd-41c2-b2d9-05658ed40106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:27.140899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e33d7-a6dd-41c2-b2d9-05658ed40106 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:27.141382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e33d7-a6dd-41c2-b2d9-05658ed40106 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:27.141717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3e33d7-a6dd-41c2-b2d9-05658ed40106 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:27.142483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e33d7-a6dd-41c2-b2d9-05658ed40106 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:27.142852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e33d7-a6dd-41c2-b2d9-05658ed40106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:27.144333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3e33d7-a6dd-41c2-b2d9-05658ed40106 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:27.149377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 108/226] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:27.151047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed8a3a-c685-4956-9fcd-5a38a4f17ed2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:27.153741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 119/227] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:27.159620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e33d7-a6dd-41c2-b2d9-05658ed40106 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:27.276425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed8a3a-c685-4956-9fcd-5a38a4f17ed2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:27.288146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3e33d7-a6dd-41c2-b2d9-05658ed40106 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:27.300405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:27.326108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b8197-a458-4414-acaf-eb3af2d3646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:27.326908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b8197-a458-4414-acaf-eb3af2d3646b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:27.327143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167b96b-fbdb-4e61-9d25-fb279b2d7a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:27.327255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b8197-a458-4414-acaf-eb3af2d3646b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:27.327579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c9b8197-a458-4414-acaf-eb3af2d3646b None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:27.328365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167b96b-fbdb-4e61-9d25-fb279b2d7a58 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:27.328804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167b96b-fbdb-4e61-9d25-fb279b2d7a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:27.329023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b8197-a458-4414-acaf-eb3af2d3646b None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:27.329402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167b96b-fbdb-4e61-9d25-fb279b2d7a58 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:27.330647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c9b8197-a458-4414-acaf-eb3af2d3646b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:27.330901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167b96b-fbdb-4e61-9d25-fb279b2d7a58 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:27.332564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167b96b-fbdb-4e61-9d25-fb279b2d7a58 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:27.332719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 120/228] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:27 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:27.333686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b8197-a458-4414-acaf-eb3af2d3646b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:27.335208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 109/229] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:27 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:27.346222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167b96b-fbdb-4e61-9d25-fb279b2d7a58 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:27.353072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:27.450234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c9b8197-a458-4414-acaf-eb3af2d3646b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:27.457757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167b96b-fbdb-4e61-9d25-fb279b2d7a58 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:27.492281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90b64c-c60e-4b2f-8409-e77627ee0915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:27.493290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90b64c-c60e-4b2f-8409-e77627ee0915 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:27.493735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90b64c-c60e-4b2f-8409-e77627ee0915 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:27.494459 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d90b64c-c60e-4b2f-8409-e77627ee0915 None admin] PATH_INFO: `/v3/users/4b1402b6b15e4995ab5c7a6789531d1a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:27.496015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90b64c-c60e-4b2f-8409-e77627ee0915 None admin] RBAC: Authorizing `identity:get_user(user_id=4b1402b6b15e4995ab5c7a6789531d1a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:27.497482 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560b99a-1343-4221-949a-bde774d647a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:27.498205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d90b64c-c60e-4b2f-8409-e77627ee0915 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:27.498357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560b99a-1343-4221-949a-bde774d647a1 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:27.498744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560b99a-1343-4221-949a-bde774d647a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:27.499078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c560b99a-1343-4221-949a-bde774d647a1 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819/users/4b1402b6b15e4995ab5c7a6789531d1a/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:27.500850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 121/230] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:27 2022] GET /identity/v3/users/4b1402b6b15e4995ab5c7a6789531d1a => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:27.501223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560b99a-1343-4221-949a-bde774d647a1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe3324999301468c8543ebc979dbd819, user_id=4b1402b6b15e4995ab5c7a6789531d1a, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:27.502053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90b64c-c60e-4b2f-8409-e77627ee0915 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:27.502831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c560b99a-1343-4221-949a-bde774d647a1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:27.523703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 110/231] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:47:27 2022] PUT /identity/v3/projects/fe3324999301468c8543ebc979dbd819/users/4b1402b6b15e4995ab5c7a6789531d1a/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:27.525128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560b99a-1343-4221-949a-bde774d647a1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:27.591425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:27.627052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:27.649023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d90b64c-c60e-4b2f-8409-e77627ee0915 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:27.677621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c560b99a-1343-4221-949a-bde774d647a1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:27.688270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814d0e-ebc5-4b3e-b78a-bb8ea1422223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:27.689289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814d0e-ebc5-4b3e-b78a-bb8ea1422223 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:27.689754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814d0e-ebc5-4b3e-b78a-bb8ea1422223 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:27.690277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71814d0e-ebc5-4b3e-b78a-bb8ea1422223 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:27.691487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71814d0e-ebc5-4b3e-b78a-bb8ea1422223 None admin] RBAC: Adding query filter params (role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=fe3324999301468c8543ebc979dbd819, user.id=3fbe94213ed84f34a7529106f60d03b8) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:27.692098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71814d0e-ebc5-4b3e-b78a-bb8ea1422223 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:27.693904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71814d0e-ebc5-4b3e-b78a-bb8ea1422223 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:27.700705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 122/232] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:27 2022] GET /identity/v3/role_assignments?user.id=3fbe94213ed84f34a7529106f60d03b8&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=2d59fcad66104464bbc848ef605f743e => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:27.725985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2a81d3-658b-4255-9276-55a8fe04d144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:27.726690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a81d3-658b-4255-9276-55a8fe04d144 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:27.726969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a81d3-658b-4255-9276-55a8fe04d144 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:27.727241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2a81d3-658b-4255-9276-55a8fe04d144 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:27.731116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a81d3-658b-4255-9276-55a8fe04d144 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:27.732844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2a81d3-658b-4255-9276-55a8fe04d144 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:27.735168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 111/233] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:27 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:27.740949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814d0e-ebc5-4b3e-b78a-bb8ea1422223 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:27.879010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71814d0e-ebc5-4b3e-b78a-bb8ea1422223 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:27.921768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa57eca-6317-4c89-9354-4703ea5a4d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:27.922495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa57eca-6317-4c89-9354-4703ea5a4d8f None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:27.922775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa57eca-6317-4c89-9354-4703ea5a4d8f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:27.923050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa57eca-6317-4c89-9354-4703ea5a4d8f None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819/users/4b1402b6b15e4995ab5c7a6789531d1a/roles/c622170611454f18bf01aa658a8cc06e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:27.925252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa57eca-6317-4c89-9354-4703ea5a4d8f None admin] RBAC: Authorizing `identity:create_grant(project_id=fe3324999301468c8543ebc979dbd819, user_id=4b1402b6b15e4995ab5c7a6789531d1a, role_id=c622170611454f18bf01aa658a8cc06e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:27.926785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa57eca-6317-4c89-9354-4703ea5a4d8f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:27.944016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 123/234] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:47:27 2022] PUT /identity/v3/projects/fe3324999301468c8543ebc979dbd819/users/4b1402b6b15e4995ab5c7a6789531d1a/roles/c622170611454f18bf01aa658a8cc06e => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:29.264935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25255907-94a2-4d21-b6cf-2222531a696a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:29.265983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25255907-94a2-4d21-b6cf-2222531a696a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:29.266492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25255907-94a2-4d21-b6cf-2222531a696a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:29.268257 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 112/235] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:29.274378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4b862-3b33-4c65-9094-f420b3a71d04 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:29.275374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4b862-3b33-4c65-9094-f420b3a71d04 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:29.275851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b4b862-3b33-4c65-9094-f420b3a71d04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:29.283926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:29.302489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d6b4b862-3b33-4c65-9094-f420b3a71d04 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:29.304407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d6b4b862-3b33-4c65-9094-f420b3a71d04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:29.391217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 124/236] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:29.438901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c3903-e671-4447-8581-b5d927743209 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:29.439210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c3903-e671-4447-8581-b5d927743209 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:29.439444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129c3903-e671-4447-8581-b5d927743209 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:29.440800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 113/237] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:29.446013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1b52d-ac0f-4c98-87ce-98a7b26cd4d5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:29.446330 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1b52d-ac0f-4c98-87ce-98a7b26cd4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:29.446600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c1b52d-ac0f-4c98-87ce-98a7b26cd4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:29.464774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-69c1b52d-ac0f-4c98-87ce-98a7b26cd4d5 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:29.466008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-69c1b52d-ac0f-4c98-87ce-98a7b26cd4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:29.549449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 125/238] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:29.803073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c3903-e671-4447-8581-b5d927743209 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:29.928386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129c3903-e671-4447-8581-b5d927743209 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:29.933220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1b52d-ac0f-4c98-87ce-98a7b26cd4d5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:29.965914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:29.970712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdfcd7-4bad-49ef-8aa8-c11e8297a427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:29.971365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdfcd7-4bad-49ef-8aa8-c11e8297a427 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:29.972022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdfcd7-4bad-49ef-8aa8-c11e8297a427 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:29.972304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cdfcd7-4bad-49ef-8aa8-c11e8297a427 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:29.984921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cdfcd7-4bad-49ef-8aa8-c11e8297a427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-13cdfcd7-4bad-49ef-8aa8-c11e8297a427 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:29.990185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:29.991050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:29.991838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 114/239] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:29.998116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdfcd7-4bad-49ef-8aa8-c11e8297a427 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:30.096548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:30.135498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cdfcd7-4bad-49ef-8aa8-c11e8297a427 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:30.173125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a81fcf-7b79-4586-ab42-5faf9cc5ced7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:30.173748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a81fcf-7b79-4586-ab42-5faf9cc5ced7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.174032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a81fcf-7b79-4586-ab42-5faf9cc5ced7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.174337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a81fcf-7b79-4586-ab42-5faf9cc5ced7 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.175261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a81fcf-7b79-4586-ab42-5faf9cc5ced7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:30.175571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a81fcf-7b79-4586-ab42-5faf9cc5ced7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:30.177046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a81fcf-7b79-4586-ab42-5faf9cc5ced7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:30.187220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 115/240] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:29 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:30.189906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c1b52d-ac0f-4c98-87ce-98a7b26cd4d5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:30.193265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a81fcf-7b79-4586-ab42-5faf9cc5ced7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:30.238750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:30.282839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931194f-7f14-4885-891d-453da7dff5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:30.283506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931194f-7f14-4885-891d-453da7dff5ff None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.283795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931194f-7f14-4885-891d-453da7dff5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931194f-7f14-4885-891d-453da7dff5ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.285289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931194f-7f14-4885-891d-453da7dff5ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:30.286788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931194f-7f14-4885-891d-453da7dff5ff None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:30.292950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 126/241] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:29 2022] GET /identity/v3/domains/default => generated 199 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:30.299115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931194f-7f14-4885-891d-453da7dff5ff None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:30.323336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a81fcf-7b79-4586-ab42-5faf9cc5ced7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:30.363379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df8e047-3504-4113-a6a5-001198e05880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:30.364244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8e047-3504-4113-a6a5-001198e05880 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.364823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8e047-3504-4113-a6a5-001198e05880 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.365236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df8e047-3504-4113-a6a5-001198e05880 None admin] PATH_INFO: `/v3/users/8b7abab7eef5492bbe7e7b0426c8971c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.381755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8e047-3504-4113-a6a5-001198e05880 None admin] RBAC: Authorizing `identity:get_user(user_id=8b7abab7eef5492bbe7e7b0426c8971c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:30.383811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df8e047-3504-4113-a6a5-001198e05880 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:30.385680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 116/242] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:30 2022] GET /identity/v3/users/8b7abab7eef5492bbe7e7b0426c8971c => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:30.387194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55e512-4d40-45af-a710-a279abfdb887 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.387612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55e512-4d40-45af-a710-a279abfdb887 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.388059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c55e512-4d40-45af-a710-a279abfdb887 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.389882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 117/243] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:30.392006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55e512-4d40-45af-a710-a279abfdb887 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:30.435137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931194f-7f14-4885-891d-453da7dff5ff None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:30.466786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:30.472159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b7fa64-e327-412e-8908-1dbf4ba318cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:30.472892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7fa64-e327-412e-8908-1dbf4ba318cb None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.473263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7fa64-e327-412e-8908-1dbf4ba318cb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.473826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b7fa64-e327-412e-8908-1dbf4ba318cb None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.474576 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7fa64-e327-412e-8908-1dbf4ba318cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:30.476950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b7fa64-e327-412e-8908-1dbf4ba318cb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:30.504525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 127/244] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:47:30 2022] POST /identity/v3/users => generated 293 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:47:30.506007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba0136-376e-441a-add2-4b13844186dc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.506819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba0136-376e-441a-add2-4b13844186dc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.507231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddba0136-376e-441a-add2-4b13844186dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.517588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:30.528317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ddba0136-376e-441a-add2-4b13844186dc None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:30.529907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ddba0136-376e-441a-add2-4b13844186dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:30.530042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c55e512-4d40-45af-a710-a279abfdb887 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:30.569686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a57c0-8427-47bf-8fac-aea092f0d154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:30.570651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a57c0-8427-47bf-8fac-aea092f0d154 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.571072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a57c0-8427-47bf-8fac-aea092f0d154 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.571321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-042a57c0-8427-47bf-8fac-aea092f0d154 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.613670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 128/245] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:30.621661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f699f1c-c0a9-4100-8012-d638c3e204d1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.622061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f699f1c-c0a9-4100-8012-d638c3e204d1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.622538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f699f1c-c0a9-4100-8012-d638c3e204d1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.624276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 129/246] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:30 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:30.630117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154772e-028d-4415-afea-0f3bf7eac645 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.630487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154772e-028d-4415-afea-0f3bf7eac645 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.630843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154772e-028d-4415-afea-0f3bf7eac645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.663358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a57c0-8427-47bf-8fac-aea092f0d154 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:30.663568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3154772e-028d-4415-afea-0f3bf7eac645 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:30.664797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3154772e-028d-4415-afea-0f3bf7eac645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:30.665399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-042a57c0-8427-47bf-8fac-aea092f0d154 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:30.765302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 130/247] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:30.766687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61982e4e-6208-4f44-9310-e2c2e44f2687 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.766967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61982e4e-6208-4f44-9310-e2c2e44f2687 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.767240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61982e4e-6208-4f44-9310-e2c2e44f2687 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.768494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 131/248] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:30.774666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3426aa-e4d3-428b-9022-e72c20a935c9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.774949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3426aa-e4d3-428b-9022-e72c20a935c9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.775227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3426aa-e4d3-428b-9022-e72c20a935c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.785565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:30.800762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 118/249] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:30.804809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-da3426aa-e4d3-428b-9022-e72c20a935c9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:30.807231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-da3426aa-e4d3-428b-9022-e72c20a935c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:30.809041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a57c0-8427-47bf-8fac-aea092f0d154 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:30.857411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:30.884879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 132/250] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:30.928883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-042a57c0-8427-47bf-8fac-aea092f0d154 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:30.946658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3426aa-e4d3-428b-9022-e72c20a935c9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:30.971207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1faaf-f4d3-4408-a8b8-45cda17564d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:30.971864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1faaf-f4d3-4408-a8b8-45cda17564d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:30.972121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1faaf-f4d3-4408-a8b8-45cda17564d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:30.972395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d1faaf-f4d3-4408-a8b8-45cda17564d2 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:30.974030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1faaf-f4d3-4408-a8b8-45cda17564d2 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:30.975878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d1faaf-f4d3-4408-a8b8-45cda17564d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:30.977621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 119/251] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:30 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:30.983487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1faaf-f4d3-4408-a8b8-45cda17564d2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.047548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:31.070358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3426aa-e4d3-428b-9022-e72c20a935c9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.095767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d1faaf-f4d3-4408-a8b8-45cda17564d2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.111084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627d4b2-fafb-4d7e-b033-2e9ac078d3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:31.112058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627d4b2-fafb-4d7e-b033-2e9ac078d3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:31.112555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627d4b2-fafb-4d7e-b033-2e9ac078d3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:31.113134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e627d4b2-fafb-4d7e-b033-2e9ac078d3d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:31.124781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e627d4b2-fafb-4d7e-b033-2e9ac078d3d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e627d4b2-fafb-4d7e-b033-2e9ac078d3d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:31.130072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:31.131003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:31.131819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:31.134422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 133/252] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:47:31.135463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdae48-d803-4d99-8b2d-109c4d4fc7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:31.135820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627d4b2-fafb-4d7e-b033-2e9ac078d3d3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.136036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdae48-d803-4d99-8b2d-109c4d4fc7dc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:31.136317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdae48-d803-4d99-8b2d-109c4d4fc7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:31.136591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfdae48-d803-4d99-8b2d-109c4d4fc7dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:31.137372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdae48-d803-4d99-8b2d-109c4d4fc7dc None admin] RBAC: Adding query filter params (role.id=15be693919a24ac69f0c9320bba5fac2, scope.project.id=fe3324999301468c8543ebc979dbd819, user.id=8b7abab7eef5492bbe7e7b0426c8971c) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:31.137632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdae48-d803-4d99-8b2d-109c4d4fc7dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:31.142106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfdae48-d803-4d99-8b2d-109c4d4fc7dc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:31.147118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 120/253] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:30 2022] GET /identity/v3/role_assignments?user.id=8b7abab7eef5492bbe7e7b0426c8971c&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=15be693919a24ac69f0c9320bba5fac2 => generated 259 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:31.147935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdae48-d803-4d99-8b2d-109c4d4fc7dc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.193631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:31.281948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e627d4b2-fafb-4d7e-b033-2e9ac078d3d3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.293426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfdae48-d803-4d99-8b2d-109c4d4fc7dc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.322721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:31.323252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1dd01-6e77-48ff-97b9-45eed9f4667e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:31.324462 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1dd01-6e77-48ff-97b9-45eed9f4667e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:31.325216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1dd01-6e77-48ff-97b9-45eed9f4667e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:31.325985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f1dd01-6e77-48ff-97b9-45eed9f4667e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:31.337320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b49a-0021-49f9-b9a3-e3f1a47b833e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:31.338033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b49a-0021-49f9-b9a3-e3f1a47b833e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:31.338718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b49a-0021-49f9-b9a3-e3f1a47b833e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:31.338718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b49a-0021-49f9-b9a3-e3f1a47b833e None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:31.339385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89b49a-0021-49f9-b9a3-e3f1a47b833e None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:31.339696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89b49a-0021-49f9-b9a3-e3f1a47b833e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:31.341141 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89b49a-0021-49f9-b9a3-e3f1a47b833e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:31.342785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f1dd01-6e77-48ff-97b9-45eed9f4667e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e0f1dd01-6e77-48ff-97b9-45eed9f4667e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:31.348191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:31.349315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:31.350322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:31.351119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 121/254] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:31.351622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b49a-0021-49f9-b9a3-e3f1a47b833e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.352775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 134/255] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:47:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:31.363683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1dd01-6e77-48ff-97b9-45eed9f4667e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.465961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b49a-0021-49f9-b9a3-e3f1a47b833e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.503671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae092c8-f6e2-499b-82f8-b275da17567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:31.504376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae092c8-f6e2-499b-82f8-b275da17567b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:31.504655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae092c8-f6e2-499b-82f8-b275da17567b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:31.504928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae092c8-f6e2-499b-82f8-b275da17567b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:31.511834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:31.516282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f1dd01-6e77-48ff-97b9-45eed9f4667e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.518653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae092c8-f6e2-499b-82f8-b275da17567b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0ae092c8-f6e2-499b-82f8-b275da17567b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:31.524092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:31.524999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:31.525784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 122/256] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:31.530696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae092c8-f6e2-499b-82f8-b275da17567b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.563398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb10030-7e3c-48cf-adb8-8145bd1f5cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:31.564297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb10030-7e3c-48cf-adb8-8145bd1f5cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:31.564861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb10030-7e3c-48cf-adb8-8145bd1f5cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:31.565273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb10030-7e3c-48cf-adb8-8145bd1f5cc4 None admin] PATH_INFO: `/v3/users/3fbe94213ed84f34a7529106f60d03b8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:31.566635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb10030-7e3c-48cf-adb8-8145bd1f5cc4 None admin] RBAC: Authorizing `identity:get_user(user_id=3fbe94213ed84f34a7529106f60d03b8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:31.568336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb10030-7e3c-48cf-adb8-8145bd1f5cc4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:31.570554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 135/257] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:31 2022] GET /identity/v3/users/3fbe94213ed84f34a7529106f60d03b8 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:31.571656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb10030-7e3c-48cf-adb8-8145bd1f5cc4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.575317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:31.639551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae092c8-f6e2-499b-82f8-b275da17567b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.680184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb10030-7e3c-48cf-adb8-8145bd1f5cc4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.707905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4576ab-a7e7-4764-b0a2-159c644fc68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:31.708914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4576ab-a7e7-4764-b0a2-159c644fc68c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:31.709163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4576ab-a7e7-4764-b0a2-159c644fc68c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:31.709534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4576ab-a7e7-4764-b0a2-159c644fc68c None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:31.710359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4576ab-a7e7-4764-b0a2-159c644fc68c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:31.710787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4576ab-a7e7-4764-b0a2-159c644fc68c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:31.712593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4576ab-a7e7-4764-b0a2-159c644fc68c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:31.725171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28210860-f43f-4886-86aa-7fa697909266 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:31.726004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28210860-f43f-4886-86aa-7fa697909266 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:31.726448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28210860-f43f-4886-86aa-7fa697909266 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:31.726997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28210860-f43f-4886-86aa-7fa697909266 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:31.728781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28210860-f43f-4886-86aa-7fa697909266 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:31.729502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28210860-f43f-4886-86aa-7fa697909266 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:31.731529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28210860-f43f-4886-86aa-7fa697909266 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:31.733168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 123/258] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:31.735009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4576ab-a7e7-4764-b0a2-159c644fc68c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.751962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 136/259] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:31.756995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28210860-f43f-4886-86aa-7fa697909266 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.810452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:31.914845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28210860-f43f-4886-86aa-7fa697909266 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.951429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e10ae1-15be-44b1-81e2-c070b04913a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:31.952084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e10ae1-15be-44b1-81e2-c070b04913a8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:31.952366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e10ae1-15be-44b1-81e2-c070b04913a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:31.952643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e10ae1-15be-44b1-81e2-c070b04913a8 None admin] PATH_INFO: `/v3/users/4b1402b6b15e4995ab5c7a6789531d1a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:31.953817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e10ae1-15be-44b1-81e2-c070b04913a8 None admin] RBAC: Authorizing `identity:get_user(user_id=4b1402b6b15e4995ab5c7a6789531d1a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:31.955411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e10ae1-15be-44b1-81e2-c070b04913a8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:31.957030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 137/260] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:31 2022] GET /identity/v3/users/4b1402b6b15e4995ab5c7a6789531d1a => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:31.957980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e10ae1-15be-44b1-81e2-c070b04913a8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:31.964428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4576ab-a7e7-4764-b0a2-159c644fc68c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:31.984305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:32.063702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e10ae1-15be-44b1-81e2-c070b04913a8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:32.064107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52050d0-e9e5-49a7-9b8d-be3d18217c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:32.064782 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52050d0-e9e5-49a7-9b8d-be3d18217c2f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:32.065059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52050d0-e9e5-49a7-9b8d-be3d18217c2f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:32.065332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52050d0-e9e5-49a7-9b8d-be3d18217c2f None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:32.066185 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:32.070811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52050d0-e9e5-49a7-9b8d-be3d18217c2f None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:32.072388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52050d0-e9e5-49a7-9b8d-be3d18217c2f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:32.074477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 124/261] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:31 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:32.075196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52050d0-e9e5-49a7-9b8d-be3d18217c2f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:32.098335 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509fbc1-3b81-46a1-8cd6-e64332cf7c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:32.098989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509fbc1-3b81-46a1-8cd6-e64332cf7c80 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:32.099271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509fbc1-3b81-46a1-8cd6-e64332cf7c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:32.099546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509fbc1-3b81-46a1-8cd6-e64332cf7c80 None admin] PATH_INFO: `/v3/users/4b1402b6b15e4995ab5c7a6789531d1a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:32.107273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509fbc1-3b81-46a1-8cd6-e64332cf7c80 None admin] RBAC: Authorizing `identity:get_user(user_id=4b1402b6b15e4995ab5c7a6789531d1a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:32.108808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509fbc1-3b81-46a1-8cd6-e64332cf7c80 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:32.110805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 138/262] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:31 2022] GET /identity/v3/users/4b1402b6b15e4995ab5c7a6789531d1a => generated 283 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:32.116219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509fbc1-3b81-46a1-8cd6-e64332cf7c80 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:32.277313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509fbc1-3b81-46a1-8cd6-e64332cf7c80 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:32.313398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab803b-8ccd-42f6-b4f0-c66497652c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:32.314071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab803b-8ccd-42f6-b4f0-c66497652c54 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:32.314394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab803b-8ccd-42f6-b4f0-c66497652c54 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:32.314672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fab803b-8ccd-42f6-b4f0-c66497652c54 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819/users/3fbe94213ed84f34a7529106f60d03b8/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:32.316619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab803b-8ccd-42f6-b4f0-c66497652c54 None admin] RBAC: Authorizing `identity:create_grant(project_id=fe3324999301468c8543ebc979dbd819, user_id=3fbe94213ed84f34a7529106f60d03b8, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:32.318096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fab803b-8ccd-42f6-b4f0-c66497652c54 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:32.332348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52050d0-e9e5-49a7-9b8d-be3d18217c2f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:32.336827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 139/263] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:47:32 2022] PUT /identity/v3/projects/fe3324999301468c8543ebc979dbd819/users/3fbe94213ed84f34a7529106f60d03b8/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:32.338775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab803b-8ccd-42f6-b4f0-c66497652c54 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:32.372069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:32.445580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bce8a4-3433-478f-ab9d-baf5660ce318 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:32.446390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bce8a4-3433-478f-ab9d-baf5660ce318 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:32.446504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bce8a4-3433-478f-ab9d-baf5660ce318 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:32.446827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bce8a4-3433-478f-ab9d-baf5660ce318 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:32.460116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fab803b-8ccd-42f6-b4f0-c66497652c54 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:32.500943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2390cb-38b4-482c-a185-df187d519d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:32.501567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2390cb-38b4-482c-a185-df187d519d29 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:32.501845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2390cb-38b4-482c-a185-df187d519d29 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:32.502149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2390cb-38b4-482c-a185-df187d519d29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:32.546256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:32.599940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bce8a4-3433-478f-ab9d-baf5660ce318 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:32.601440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54bce8a4-3433-478f-ab9d-baf5660ce318 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:32.660092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2390cb-38b4-482c-a185-df187d519d29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:32.661610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2390cb-38b4-482c-a185-df187d519d29 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:32.694869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:32.810377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 125/264] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 735 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:32.819612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bce8a4-3433-478f-ab9d-baf5660ce318 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:32.822943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 140/265] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 485 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:32.837346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2390cb-38b4-482c-a185-df187d519d29 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:32.988522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:33.012663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bce8a4-3433-478f-ab9d-baf5660ce318 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:33.023491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2390cb-38b4-482c-a185-df187d519d29 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:33.050531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b55ae-76a0-4081-a53b-9c69d50acb05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:33.051609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b55ae-76a0-4081-a53b-9c69d50acb05 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:33.052890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b55ae-76a0-4081-a53b-9c69d50acb05 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:33.054179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1b55ae-76a0-4081-a53b-9c69d50acb05 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:33.056665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b55ae-76a0-4081-a53b-9c69d50acb05 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:33.058722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1b55ae-76a0-4081-a53b-9c69d50acb05 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:33.061976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 126/266] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:32 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:33.072029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b55ae-76a0-4081-a53b-9c69d50acb05 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:33.077131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:33.102173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c4166-bb16-47cf-8b47-9f91f1af4c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:33.103565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c4166-bb16-47cf-8b47-9f91f1af4c6d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:33.104233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c4166-bb16-47cf-8b47-9f91f1af4c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:33.104813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820c4166-bb16-47cf-8b47-9f91f1af4c6d None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:33.107669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c4166-bb16-47cf-8b47-9f91f1af4c6d None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:33.110114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820c4166-bb16-47cf-8b47-9f91f1af4c6d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:33.118117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 141/267] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:32 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:33.120106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c4166-bb16-47cf-8b47-9f91f1af4c6d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:33.282911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1b55ae-76a0-4081-a53b-9c69d50acb05 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:33.299911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820c4166-bb16-47cf-8b47-9f91f1af4c6d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:33.322100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3ec699-a2b8-4047-9068-96a16148fc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:33.322647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ec699-a2b8-4047-9068-96a16148fc60 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:33.322930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ec699-a2b8-4047-9068-96a16148fc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:33.323202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3ec699-a2b8-4047-9068-96a16148fc60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:33.323948 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ec699-a2b8-4047-9068-96a16148fc60 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=fe3324999301468c8543ebc979dbd819, user.id=4b1402b6b15e4995ab5c7a6789531d1a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:33.324257 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ec699-a2b8-4047-9068-96a16148fc60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:33.326628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3ec699-a2b8-4047-9068-96a16148fc60 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:33.340453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 127/268] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:33 2022] GET /identity/v3/role_assignments?user.id=4b1402b6b15e4995ab5c7a6789531d1a&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 608 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:33.352316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a462d13-59bb-4671-8291-73eaab3f748e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:33.353060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a462d13-59bb-4671-8291-73eaab3f748e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:33.353220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a462d13-59bb-4671-8291-73eaab3f748e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:33.353496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a462d13-59bb-4671-8291-73eaab3f748e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:33.354450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a462d13-59bb-4671-8291-73eaab3f748e None admin] RBAC: Adding query filter params (role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=fe3324999301468c8543ebc979dbd819, user.id=4b1402b6b15e4995ab5c7a6789531d1a) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:33.354767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a462d13-59bb-4671-8291-73eaab3f748e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:33.356243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a462d13-59bb-4671-8291-73eaab3f748e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:33.363279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 142/269] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:33 2022] GET /identity/v3/role_assignments?user.id=4b1402b6b15e4995ab5c7a6789531d1a&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=c622170611454f18bf01aa658a8cc06e => generated 608 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:33.664631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7366605-3bea-479d-8321-6d744222105c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:33.665204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7366605-3bea-479d-8321-6d744222105c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:33.665204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7366605-3bea-479d-8321-6d744222105c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:33.666960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 128/270] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:33.676319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb7d7e-608e-4320-aff9-9087ce6c3c8e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:33.676634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb7d7e-608e-4320-aff9-9087ce6c3c8e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:33.676879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cb7d7e-608e-4320-aff9-9087ce6c3c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:33.697964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-11cb7d7e-608e-4320-aff9-9087ce6c3c8e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:33.699591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11cb7d7e-608e-4320-aff9-9087ce6c3c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:33.759736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:33.783461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 143/271] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:33.953715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11801ec-2a3e-49f7-b7fc-b3881b0983f6 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:33.953903 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11801ec-2a3e-49f7-b7fc-b3881b0983f6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:33.954216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11801ec-2a3e-49f7-b7fc-b3881b0983f6 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:33.955704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 129/272] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:33.961278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b979f7d-a905-4b0c-8396-c3b7ecb1d416 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:33.961796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b979f7d-a905-4b0c-8396-c3b7ecb1d416 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:33.962333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b979f7d-a905-4b0c-8396-c3b7ecb1d416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:33.983018 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8b979f7d-a905-4b0c-8396-c3b7ecb1d416 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:33.984523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8b979f7d-a905-4b0c-8396-c3b7ecb1d416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:34.058633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 144/273] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:34.102752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11801ec-2a3e-49f7-b7fc-b3881b0983f6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:34.116390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:34.213998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11801ec-2a3e-49f7-b7fc-b3881b0983f6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:34.225859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96d570-77f1-4593-9a8e-8e722c0d1ffe None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:34.226192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96d570-77f1-4593-9a8e-8e722c0d1ffe None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:34.226466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a96d570-77f1-4593-9a8e-8e722c0d1ffe None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:34.227818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 145/274] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:34.232726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eece80-e792-4a97-bad1-729f1dca5ae8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:34.233000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eece80-e792-4a97-bad1-729f1dca5ae8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:34.233271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65eece80-e792-4a97-bad1-729f1dca5ae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:34.249705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e351f9a-1a56-41de-98ab-ae957701ff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:34.250383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e351f9a-1a56-41de-98ab-ae957701ff1a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:34.250666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e351f9a-1a56-41de-98ab-ae957701ff1a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:34.250942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e351f9a-1a56-41de-98ab-ae957701ff1a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:34.251942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-65eece80-e792-4a97-bad1-729f1dca5ae8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:34.252053 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e351f9a-1a56-41de-98ab-ae957701ff1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:34.253122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-65eece80-e792-4a97-bad1-729f1dca5ae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:34.253514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e351f9a-1a56-41de-98ab-ae957701ff1a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:34.255437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 130/275] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:34.260970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e351f9a-1a56-41de-98ab-ae957701ff1a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:34.337619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 146/276] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:34.338594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eece80-e792-4a97-bad1-729f1dca5ae8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:34.339395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:34.363150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e351f9a-1a56-41de-98ab-ae957701ff1a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:34.392392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:34.398297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6525cd9-5c86-49a1-9991-4a0ad1f4295a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:34.398840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6525cd9-5c86-49a1-9991-4a0ad1f4295a None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:34.399123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6525cd9-5c86-49a1-9991-4a0ad1f4295a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:34.399401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6525cd9-5c86-49a1-9991-4a0ad1f4295a None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:34.400062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6525cd9-5c86-49a1-9991-4a0ad1f4295a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:34.401547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6525cd9-5c86-49a1-9991-4a0ad1f4295a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:34.434307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 131/277] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:47:34 2022] POST /identity/v3/users => generated 293 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:47:34.457080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65eece80-e792-4a97-bad1-729f1dca5ae8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:34.498931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6635416-d693-49b9-bb3e-31e13f1fe648 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:34.499832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6635416-d693-49b9-bb3e-31e13f1fe648 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:34.500249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6635416-d693-49b9-bb3e-31e13f1fe648 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:34.500707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6635416-d693-49b9-bb3e-31e13f1fe648 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:34.513106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6635416-d693-49b9-bb3e-31e13f1fe648 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c6635416-d693-49b9-bb3e-31e13f1fe648 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:34.518446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:34.519529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:34.520511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:34.522623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 147/278] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:34.528775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6525cd9-5c86-49a1-9991-4a0ad1f4295a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:34.612232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6635416-d693-49b9-bb3e-31e13f1fe648 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:34.643739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6525cd9-5c86-49a1-9991-4a0ad1f4295a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:34.675068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:34.682679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4320ee-eb90-4be9-8881-8940b3b00da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:34.683794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4320ee-eb90-4be9-8881-8940b3b00da0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:34.684177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4320ee-eb90-4be9-8881-8940b3b00da0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:34.684557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4320ee-eb90-4be9-8881-8940b3b00da0 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:34.686276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4320ee-eb90-4be9-8881-8940b3b00da0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:34.687602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4320ee-eb90-4be9-8881-8940b3b00da0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:34.689177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4320ee-eb90-4be9-8881-8940b3b00da0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:34.701320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 132/279] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:34.710293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4320ee-eb90-4be9-8881-8940b3b00da0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:34.732973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6635416-d693-49b9-bb3e-31e13f1fe648 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:34.745727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:34.782143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d17e38-3b6b-49c8-b397-9270e395b52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:34.785890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d17e38-3b6b-49c8-b397-9270e395b52c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:34.786203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d17e38-3b6b-49c8-b397-9270e395b52c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:34.786485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d17e38-3b6b-49c8-b397-9270e395b52c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:34.798066 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d17e38-3b6b-49c8-b397-9270e395b52c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d8d17e38-3b6b-49c8-b397-9270e395b52c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:34.804292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:34.805198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:34.806437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 148/280] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:47:34.812715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d17e38-3b6b-49c8-b397-9270e395b52c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:34.834182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4320ee-eb90-4be9-8881-8940b3b00da0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:34.879134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43a985-796a-458c-ac5a-a257ead31b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:34.879810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43a985-796a-458c-ac5a-a257ead31b01 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:34.880088 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43a985-796a-458c-ac5a-a257ead31b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:34.880353 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f43a985-796a-458c-ac5a-a257ead31b01 None admin] PATH_INFO: `/v3/users/55ad0ae539644ddcb274eb6ccfb3c4aa` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:34.896326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43a985-796a-458c-ac5a-a257ead31b01 None admin] RBAC: Authorizing `identity:get_user(user_id=55ad0ae539644ddcb274eb6ccfb3c4aa)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:34.898668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f43a985-796a-458c-ac5a-a257ead31b01 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:34.900546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 133/281] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:34 2022] GET /identity/v3/users/55ad0ae539644ddcb274eb6ccfb3c4aa => generated 293 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:34.906695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43a985-796a-458c-ac5a-a257ead31b01 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:34.941227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d17e38-3b6b-49c8-b397-9270e395b52c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:34.964855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:34.982256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043b512-6a25-411d-a689-4518c28b2ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:34.983046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043b512-6a25-411d-a689-4518c28b2ed3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:34.983483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043b512-6a25-411d-a689-4518c28b2ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:34.983826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0043b512-6a25-411d-a689-4518c28b2ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:34.984534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043b512-6a25-411d-a689-4518c28b2ed3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:34.984928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043b512-6a25-411d-a689-4518c28b2ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:34.986437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0043b512-6a25-411d-a689-4518c28b2ed3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:34.996391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 149/282] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:35.006780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043b512-6a25-411d-a689-4518c28b2ed3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:35.037540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f43a985-796a-458c-ac5a-a257ead31b01 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:35.062181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:35.104097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297c407-e5e3-4f2b-b0a1-5a96b7443758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:35.105828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297c407-e5e3-4f2b-b0a1-5a96b7443758 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:35.106125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297c407-e5e3-4f2b-b0a1-5a96b7443758 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:35.106403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3297c407-e5e3-4f2b-b0a1-5a96b7443758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:35.107322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297c407-e5e3-4f2b-b0a1-5a96b7443758 None admin] RBAC: Adding query filter params (role.id=2d59fcad66104464bbc848ef605f743e, user.id=55ad0ae539644ddcb274eb6ccfb3c4aa, scope.system=all) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:35.107629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297c407-e5e3-4f2b-b0a1-5a96b7443758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:35.109082 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3297c407-e5e3-4f2b-b0a1-5a96b7443758 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:35.119102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 134/283] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Sep 13 06:47:34 2022] GET /identity/v3/role_assignments?user.id=55ad0ae539644ddcb274eb6ccfb3c4aa&scope.system=all&role.id=2d59fcad66104464bbc848ef605f743e => generated 226 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:35.177652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0043b512-6a25-411d-a689-4518c28b2ed3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:35.230926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05030f47-184d-4b2a-a757-6ee98a85e5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:35.231675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05030f47-184d-4b2a-a757-6ee98a85e5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:35.232137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05030f47-184d-4b2a-a757-6ee98a85e5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:35.232986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05030f47-184d-4b2a-a757-6ee98a85e5f0 None admin] PATH_INFO: `/v3/users/8b7abab7eef5492bbe7e7b0426c8971c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:35.234293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05030f47-184d-4b2a-a757-6ee98a85e5f0 None admin] RBAC: Authorizing `identity:get_user(user_id=8b7abab7eef5492bbe7e7b0426c8971c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:35.235793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05030f47-184d-4b2a-a757-6ee98a85e5f0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:35.237445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 150/284] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:35 2022] GET /identity/v3/users/8b7abab7eef5492bbe7e7b0426c8971c => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:35.243255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297c407-e5e3-4f2b-b0a1-5a96b7443758 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:35.331451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16fb7c-3cec-4bed-9666-c4ab80f6866b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:35.331748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16fb7c-3cec-4bed-9666-c4ab80f6866b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:35.331905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b16fb7c-3cec-4bed-9666-c4ab80f6866b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:35.333303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 151/285] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:47:35.339719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1a28c-d9e7-4c52-8de7-5b6e2eb14e42 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:35.340280 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1a28c-d9e7-4c52-8de7-5b6e2eb14e42 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:35.340742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe1a28c-d9e7-4c52-8de7-5b6e2eb14e42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:35.360861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1fe1a28c-d9e7-4c52-8de7-5b6e2eb14e42 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:35.362464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1fe1a28c-d9e7-4c52-8de7-5b6e2eb14e42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:35.365055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3297c407-e5e3-4f2b-b0a1-5a96b7443758 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:35.386115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:35.401118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be73741-77f6-4cec-b827-206e031577f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:35.401763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be73741-77f6-4cec-b827-206e031577f5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:35.402044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be73741-77f6-4cec-b827-206e031577f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:35.402352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be73741-77f6-4cec-b827-206e031577f5 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:35.403520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be73741-77f6-4cec-b827-206e031577f5 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:35.405038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be73741-77f6-4cec-b827-206e031577f5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:35.410103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 135/286] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:35 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:35.413282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be73741-77f6-4cec-b827-206e031577f5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:35.521575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be73741-77f6-4cec-b827-206e031577f5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:35.541655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 152/287] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:35.566459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec3181-8cc1-4cdd-a51f-1c4bc68fbe0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:35.567139 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec3181-8cc1-4cdd-a51f-1c4bc68fbe0b None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:35.567422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec3181-8cc1-4cdd-a51f-1c4bc68fbe0b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:35.567702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ec3181-8cc1-4cdd-a51f-1c4bc68fbe0b None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819/users/8b7abab7eef5492bbe7e7b0426c8971c/roles/15be693919a24ac69f0c9320bba5fac2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:35.569651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec3181-8cc1-4cdd-a51f-1c4bc68fbe0b None admin] RBAC: Authorizing `identity:create_grant(project_id=fe3324999301468c8543ebc979dbd819, user_id=8b7abab7eef5492bbe7e7b0426c8971c, role_id=15be693919a24ac69f0c9320bba5fac2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:35.571207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ec3181-8cc1-4cdd-a51f-1c4bc68fbe0b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:35.594303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 136/288] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:47:35 2022] PUT /identity/v3/projects/fe3324999301468c8543ebc979dbd819/users/8b7abab7eef5492bbe7e7b0426c8971c/roles/15be693919a24ac69f0c9320bba5fac2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 06:47:35.911390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1a28c-d9e7-4c52-8de7-5b6e2eb14e42 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:35.949836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:36.037685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe1a28c-d9e7-4c52-8de7-5b6e2eb14e42 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:36.084908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c7354-165b-4987-a50b-fab57297137a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:36.085565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c7354-165b-4987-a50b-fab57297137a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:36.085848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c7354-165b-4987-a50b-fab57297137a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:36.086172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c7354-165b-4987-a50b-fab57297137a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:36.103129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c7354-165b-4987-a50b-fab57297137a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-006c7354-165b-4987-a50b-fab57297137a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:36.108249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:36.109145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:36.109925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 153/289] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:47:36.115234 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec3181-8cc1-4cdd-a51f-1c4bc68fbe0b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:36.162879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:36.221650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ec3181-8cc1-4cdd-a51f-1c4bc68fbe0b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:36.260892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de24bc4-b54b-40f1-9859-8701572b0e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:36.261600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de24bc4-b54b-40f1-9859-8701572b0e62 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:36.261887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de24bc4-b54b-40f1-9859-8701572b0e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:36.262190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de24bc4-b54b-40f1-9859-8701572b0e62 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:36.262903 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de24bc4-b54b-40f1-9859-8701572b0e62 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:36.263218 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de24bc4-b54b-40f1-9859-8701572b0e62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:36.264710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de24bc4-b54b-40f1-9859-8701572b0e62 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:36.273990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 137/290] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:36.286625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c7354-165b-4987-a50b-fab57297137a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:36.398554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:36.421015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c7354-165b-4987-a50b-fab57297137a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:36.460712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eed13a-9621-4307-b939-cb740f09bc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:36.461356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eed13a-9621-4307-b939-cb740f09bc91 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:36.461637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eed13a-9621-4307-b939-cb740f09bc91 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:36.461912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9eed13a-9621-4307-b939-cb740f09bc91 None admin] PATH_INFO: `/v3/users/3fbe94213ed84f34a7529106f60d03b8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:36.463141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eed13a-9621-4307-b939-cb740f09bc91 None admin] RBAC: Authorizing `identity:get_user(user_id=3fbe94213ed84f34a7529106f60d03b8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:36.464608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9eed13a-9621-4307-b939-cb740f09bc91 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:36.466743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 154/291] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:36 2022] GET /identity/v3/users/3fbe94213ed84f34a7529106f60d03b8 => generated 297 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:36.471960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de24bc4-b54b-40f1-9859-8701572b0e62 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:36.634592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de24bc4-b54b-40f1-9859-8701572b0e62 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:36.648368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:36.688362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15ae7-7df6-41a9-8723-55f157d23724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:36.688895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15ae7-7df6-41a9-8723-55f157d23724 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:36.689178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15ae7-7df6-41a9-8723-55f157d23724 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:36.689451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d15ae7-7df6-41a9-8723-55f157d23724 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:36.787980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d15ae7-7df6-41a9-8723-55f157d23724 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:36.789459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d15ae7-7df6-41a9-8723-55f157d23724 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:36.920737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 138/292] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:36.927024 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eed13a-9621-4307-b939-cb740f09bc91 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:37.052122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9eed13a-9621-4307-b939-cb740f09bc91 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:37.103325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9520fa47-891e-4ed3-b758-db99d13056d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:37.103971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520fa47-891e-4ed3-b758-db99d13056d8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:37.104253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520fa47-891e-4ed3-b758-db99d13056d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:37.104519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9520fa47-891e-4ed3-b758-db99d13056d8 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:37.105635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520fa47-891e-4ed3-b758-db99d13056d8 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:37.107234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9520fa47-891e-4ed3-b758-db99d13056d8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:37.109246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 155/293] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:36 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:37.115208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15ae7-7df6-41a9-8723-55f157d23724 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:37.123859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:37.224483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbb1a2-6254-4996-b765-a47941476d93 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:37.224823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbb1a2-6254-4996-b765-a47941476d93 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:37.225058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bbb1a2-6254-4996-b765-a47941476d93 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:37.226748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 156/294] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:37.228011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d15ae7-7df6-41a9-8723-55f157d23724 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:37.231491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2fd24-d82a-42e8-81ec-76b1ea355140 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:37.231840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2fd24-d82a-42e8-81ec-76b1ea355140 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:37.232120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e2fd24-d82a-42e8-81ec-76b1ea355140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:37.264131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-52e2fd24-d82a-42e8-81ec-76b1ea355140 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:37.264614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11979fab-8412-4b72-9e2a-2c160464561f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:37.265246 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11979fab-8412-4b72-9e2a-2c160464561f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:37.265530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11979fab-8412-4b72-9e2a-2c160464561f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:37.265758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-52e2fd24-d82a-42e8-81ec-76b1ea355140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:37.265858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11979fab-8412-4b72-9e2a-2c160464561f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:37.266564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11979fab-8412-4b72-9e2a-2c160464561f None admin] RBAC: Adding query filter params (role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=fe3324999301468c8543ebc979dbd819, user.id=3fbe94213ed84f34a7529106f60d03b8) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:37.266880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11979fab-8412-4b72-9e2a-2c160464561f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:37.268332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11979fab-8412-4b72-9e2a-2c160464561f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:37.277107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 139/295] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:37 2022] GET /identity/v3/role_assignments?user.id=3fbe94213ed84f34a7529106f60d03b8&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=2d59fcad66104464bbc848ef605f743e => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:37.303664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:37.377373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 157/296] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:37.601329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e011d69-b6b2-42a4-b8df-14cc539ed1a3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:37.602005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e011d69-b6b2-42a4-b8df-14cc539ed1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:37.602513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e011d69-b6b2-42a4-b8df-14cc539ed1a3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:37.604314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 140/297] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:37.615611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183de5-9f3b-42f7-8385-8a12e873884b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:37.616339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183de5-9f3b-42f7-8385-8a12e873884b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:37.616805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183de5-9f3b-42f7-8385-8a12e873884b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:37.638008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-86183de5-9f3b-42f7-8385-8a12e873884b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:37.640347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-86183de5-9f3b-42f7-8385-8a12e873884b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:37.711315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 158/298] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:37.745538 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e011d69-b6b2-42a4-b8df-14cc539ed1a3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:37.830200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:37.869340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e011d69-b6b2-42a4-b8df-14cc539ed1a3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:37.906987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936c0c5-d74b-431c-9f7e-70a376d3a6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:37.908013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936c0c5-d74b-431c-9f7e-70a376d3a6b1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:37.908485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936c0c5-d74b-431c-9f7e-70a376d3a6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:37.908978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7936c0c5-d74b-431c-9f7e-70a376d3a6b1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:37.919636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949d7f4-4472-4db6-9767-b87748d5dc07 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:37.920270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949d7f4-4472-4db6-9767-b87748d5dc07 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:37.920601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949d7f4-4472-4db6-9767-b87748d5dc07 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:37.921904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7936c0c5-d74b-431c-9f7e-70a376d3a6b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:37.922298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 159/299] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:47:37.927275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794d44a-21db-4f46-bf6a-bd9dd44c8f89 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:37.927838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794d44a-21db-4f46-bf6a-bd9dd44c8f89 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:37.927838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794d44a-21db-4f46-bf6a-bd9dd44c8f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7936c0c5-d74b-431c-9f7e-70a376d3a6b1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:37.928214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:37.929040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:37.929877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:37.932170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 141/300] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:37.938161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936c0c5-d74b-431c-9f7e-70a376d3a6b1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:37.959327 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8794d44a-21db-4f46-bf6a-bd9dd44c8f89 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:37.960703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8794d44a-21db-4f46-bf6a-bd9dd44c8f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:38.032338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 160/301] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:38.044532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794d44a-21db-4f46-bf6a-bd9dd44c8f89 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:38.058937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7936c0c5-d74b-431c-9f7e-70a376d3a6b1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:38.060021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:38.092009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:38.099932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425007c5-32c4-4493-94f5-af10a86b9dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:38.100555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425007c5-32c4-4493-94f5-af10a86b9dcd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.100928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425007c5-32c4-4493-94f5-af10a86b9dcd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.101199 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425007c5-32c4-4493-94f5-af10a86b9dcd None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.101980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425007c5-32c4-4493-94f5-af10a86b9dcd None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:38.102308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425007c5-32c4-4493-94f5-af10a86b9dcd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:38.103841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425007c5-32c4-4493-94f5-af10a86b9dcd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:38.112822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 142/302] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:38.118541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425007c5-32c4-4493-94f5-af10a86b9dcd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:38.156478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794d44a-21db-4f46-bf6a-bd9dd44c8f89 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:38.198716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc993005-adca-4b3b-9338-850c5f991c68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:38.199480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc993005-adca-4b3b-9338-850c5f991c68 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.199831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc993005-adca-4b3b-9338-850c5f991c68 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.200241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc993005-adca-4b3b-9338-850c5f991c68 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.201698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc993005-adca-4b3b-9338-850c5f991c68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:38.203191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc993005-adca-4b3b-9338-850c5f991c68 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:38.209620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 161/303] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:38.210586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc993005-adca-4b3b-9338-850c5f991c68 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:38.229599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425007c5-32c4-4493-94f5-af10a86b9dcd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:38.273790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199c9fd3-9c34-4307-8565-8ac700fdd5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:38.274592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c9fd3-9c34-4307-8565-8ac700fdd5ff None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.274876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c9fd3-9c34-4307-8565-8ac700fdd5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.275304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199c9fd3-9c34-4307-8565-8ac700fdd5ff None admin] PATH_INFO: `/v3/users/1ccd550aa2cb4504862efd42b8104f1c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.290028 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c9fd3-9c34-4307-8565-8ac700fdd5ff None admin] RBAC: Authorizing `identity:get_user(user_id=1ccd550aa2cb4504862efd42b8104f1c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:38.291540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199c9fd3-9c34-4307-8565-8ac700fdd5ff None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:38.298277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 143/304] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:38 2022] GET /identity/v3/users/1ccd550aa2cb4504862efd42b8104f1c => generated 293 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:38.298277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637dd7d-2d96-4ab4-8634-41df9c46a315 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.298277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637dd7d-2d96-4ab4-8634-41df9c46a315 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.298277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637dd7d-2d96-4ab4-8634-41df9c46a315 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.298277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 144/305] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:38.299219 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637dd7d-2d96-4ab4-8634-41df9c46a315 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:38.303721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:38.326625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc993005-adca-4b3b-9338-850c5f991c68 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:38.351184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:38.359888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88ca27a-4e44-4764-9616-95e9d6516982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:38.360509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ca27a-4e44-4764-9616-95e9d6516982 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.360789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ca27a-4e44-4764-9616-95e9d6516982 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.361063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88ca27a-4e44-4764-9616-95e9d6516982 None admin] PATH_INFO: `/v3/groups` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.361700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ca27a-4e44-4764-9616-95e9d6516982 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:38.363197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88ca27a-4e44-4764-9616-95e9d6516982 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:38.380232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 162/306] 10.222.0.25 () {60 vars in 1134 bytes} [Tue Sep 13 06:47:38 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:47:38.381338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd85d6f-a7bf-4d37-8380-3a134b0de136 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.381631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd85d6f-a7bf-4d37-8380-3a134b0de136 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.381904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd85d6f-a7bf-4d37-8380-3a134b0de136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.400855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9bd85d6f-a7bf-4d37-8380-3a134b0de136 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:38.402117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9bd85d6f-a7bf-4d37-8380-3a134b0de136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:38.410111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637dd7d-2d96-4ab4-8634-41df9c46a315 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:38.451368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30157e1b-9c79-4490-9ced-7d2109128966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:38.451999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30157e1b-9c79-4490-9ced-7d2109128966 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.452440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30157e1b-9c79-4490-9ced-7d2109128966 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.452721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30157e1b-9c79-4490-9ced-7d2109128966 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.453463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30157e1b-9c79-4490-9ced-7d2109128966 None admin] RBAC: Adding query filter params (role.id=15be693919a24ac69f0c9320bba5fac2, user.id=1ccd550aa2cb4504862efd42b8104f1c, scope.system=all) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:38.453774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30157e1b-9c79-4490-9ced-7d2109128966 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:38.455407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30157e1b-9c79-4490-9ced-7d2109128966 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:38.474464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 145/307] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Sep 13 06:47:38 2022] GET /identity/v3/role_assignments?user.id=1ccd550aa2cb4504862efd42b8104f1c&scope.system=all&role.id=15be693919a24ac69f0c9320bba5fac2 => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:38.475313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30157e1b-9c79-4490-9ced-7d2109128966 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:38.483233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 163/308] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:38.614325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30157e1b-9c79-4490-9ced-7d2109128966 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:38.631150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:38.657023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ca260-2628-44f0-8aaf-12831392b5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:38.658067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ca260-2628-44f0-8aaf-12831392b5cc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.658680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ca260-2628-44f0-8aaf-12831392b5cc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.659146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-617ca260-2628-44f0-8aaf-12831392b5cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.670958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-617ca260-2628-44f0-8aaf-12831392b5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-617ca260-2628-44f0-8aaf-12831392b5cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:38.676727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:38.677610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:38.678454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:38.681087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 146/309] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:38.688922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd85d6f-a7bf-4d37-8380-3a134b0de136 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:38.753520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ca260-2628-44f0-8aaf-12831392b5cc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:38.764644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:38.803457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd85d6f-a7bf-4d37-8380-3a134b0de136 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:38.841796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e8e92-9ab2-4d63-a34f-f34b7d044b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:38.842481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e8e92-9ab2-4d63-a34f-f34b7d044b9c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.842773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e8e92-9ab2-4d63-a34f-f34b7d044b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.843030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6e8e92-9ab2-4d63-a34f-f34b7d044b9c None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.843755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e8e92-9ab2-4d63-a34f-f34b7d044b9c None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:38.844055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e8e92-9ab2-4d63-a34f-f34b7d044b9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:38.845497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6e8e92-9ab2-4d63-a34f-f34b7d044b9c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:38.859404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 164/310] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:38.860917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e8e92-9ab2-4d63-a34f-f34b7d044b9c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:38.879487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-617ca260-2628-44f0-8aaf-12831392b5cc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:38.914771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1b5e9-deb2-488c-8f13-88c0a982de25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:38.915520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1b5e9-deb2-488c-8f13-88c0a982de25 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:38.915856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1b5e9-deb2-488c-8f13-88c0a982de25 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:38.916190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1b5e9-deb2-488c-8f13-88c0a982de25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:38.930387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1b5e9-deb2-488c-8f13-88c0a982de25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d5f1b5e9-deb2-488c-8f13-88c0a982de25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:38.937253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:38.938370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:38.939362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:38.945012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 147/311] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:47:38.945953 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1b5e9-deb2-488c-8f13-88c0a982de25 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:38.973271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6e8e92-9ab2-4d63-a34f-f34b7d044b9c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:38.988831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:39.010560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:39.018118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd381d-3a24-47d0-ac20-370aa98c089a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:39.019123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd381d-3a24-47d0-ac20-370aa98c089a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:39.019830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd381d-3a24-47d0-ac20-370aa98c089a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:39.020271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bd381d-3a24-47d0-ac20-370aa98c089a None admin] PATH_INFO: `/v3/users/55ad0ae539644ddcb274eb6ccfb3c4aa` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:39.021936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd381d-3a24-47d0-ac20-370aa98c089a None admin] RBAC: Authorizing `identity:get_user(user_id=55ad0ae539644ddcb274eb6ccfb3c4aa)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:39.024126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bd381d-3a24-47d0-ac20-370aa98c089a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:39.025871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 165/312] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:38 2022] GET /identity/v3/users/55ad0ae539644ddcb274eb6ccfb3c4aa => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:39.032512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd381d-3a24-47d0-ac20-370aa98c089a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:39.072806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1b5e9-deb2-488c-8f13-88c0a982de25 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:39.111210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c458c-4105-4aa4-8f79-ff9a3abd1a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:39.112073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c458c-4105-4aa4-8f79-ff9a3abd1a00 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:39.112439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c458c-4105-4aa4-8f79-ff9a3abd1a00 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:39.112809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4c458c-4105-4aa4-8f79-ff9a3abd1a00 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:39.113577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c458c-4105-4aa4-8f79-ff9a3abd1a00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:39.113980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c458c-4105-4aa4-8f79-ff9a3abd1a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:39.115978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4c458c-4105-4aa4-8f79-ff9a3abd1a00 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:39.126215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 148/313] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:39.132139 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c458c-4105-4aa4-8f79-ff9a3abd1a00 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:39.167809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bd381d-3a24-47d0-ac20-370aa98c089a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:39.212552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e257f65-d8b8-46a9-8d2d-72b22bcd7ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:39.214170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e257f65-d8b8-46a9-8d2d-72b22bcd7ea6 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:39.214643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e257f65-d8b8-46a9-8d2d-72b22bcd7ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:39.215341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e257f65-d8b8-46a9-8d2d-72b22bcd7ea6 None admin] PATH_INFO: `/v3/system/users/55ad0ae539644ddcb274eb6ccfb3c4aa/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:39.218123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e257f65-d8b8-46a9-8d2d-72b22bcd7ea6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=55ad0ae539644ddcb274eb6ccfb3c4aa, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:39.219757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e257f65-d8b8-46a9-8d2d-72b22bcd7ea6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:39.238673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 166/314] 10.222.0.25 () {58 vars in 1334 bytes} [Tue Sep 13 06:47:39 2022] PUT /identity/v3/system/users/55ad0ae539644ddcb274eb6ccfb3c4aa/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:39.254471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:39.261945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4c458c-4105-4aa4-8f79-ff9a3abd1a00 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:39.302303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22517970-cebd-421e-9bbf-f494f20e13e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:39.302951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22517970-cebd-421e-9bbf-f494f20e13e1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:39.303229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22517970-cebd-421e-9bbf-f494f20e13e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:39.303503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22517970-cebd-421e-9bbf-f494f20e13e1 None admin] PATH_INFO: `/v3/users/8b7abab7eef5492bbe7e7b0426c8971c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:39.304702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22517970-cebd-421e-9bbf-f494f20e13e1 None admin] RBAC: Authorizing `identity:get_user(user_id=8b7abab7eef5492bbe7e7b0426c8971c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:39.306276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22517970-cebd-421e-9bbf-f494f20e13e1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:39.308690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 149/315] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:39 2022] GET /identity/v3/users/8b7abab7eef5492bbe7e7b0426c8971c => generated 297 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:39.319649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e257f65-d8b8-46a9-8d2d-72b22bcd7ea6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:39.394036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:39.440808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e257f65-d8b8-46a9-8d2d-72b22bcd7ea6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:39.476977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e465a91-1d5e-43c7-bdcb-e1a63bed1951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:39.477589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e465a91-1d5e-43c7-bdcb-e1a63bed1951 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:39.477863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e465a91-1d5e-43c7-bdcb-e1a63bed1951 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:39.478155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e465a91-1d5e-43c7-bdcb-e1a63bed1951 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:39.545735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e465a91-1d5e-43c7-bdcb-e1a63bed1951 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:39.547255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e465a91-1d5e-43c7-bdcb-e1a63bed1951 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:39.608090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:39.654543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 167/316] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:39.659091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22517970-cebd-421e-9bbf-f494f20e13e1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:39.764543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22517970-cebd-421e-9bbf-f494f20e13e1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:39.800093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c7f28c-cc1f-4fef-b2df-930a851f072d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:39.801077 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f28c-cc1f-4fef-b2df-930a851f072d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:39.801546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f28c-cc1f-4fef-b2df-930a851f072d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:39.802001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c7f28c-cc1f-4fef-b2df-930a851f072d None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:39.803334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7f28c-cc1f-4fef-b2df-930a851f072d None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:39.805013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c7f28c-cc1f-4fef-b2df-930a851f072d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:39.807171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 150/317] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:39 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:39.813847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e465a91-1d5e-43c7-bdcb-e1a63bed1951 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:39.919841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e465a91-1d5e-43c7-bdcb-e1a63bed1951 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:39.955414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1e2750-5e02-425d-8f27-89c7db0a1eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:39.956023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e2750-5e02-425d-8f27-89c7db0a1eb5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:39.956295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e2750-5e02-425d-8f27-89c7db0a1eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:39.956561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1e2750-5e02-425d-8f27-89c7db0a1eb5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:39.957283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e2750-5e02-425d-8f27-89c7db0a1eb5 None admin] RBAC: Adding query filter params (role.id=15be693919a24ac69f0c9320bba5fac2, scope.project.id=fe3324999301468c8543ebc979dbd819, user.id=8b7abab7eef5492bbe7e7b0426c8971c) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:39.957585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e2750-5e02-425d-8f27-89c7db0a1eb5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:39.959136 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1e2750-5e02-425d-8f27-89c7db0a1eb5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:39.965020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 168/318] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:47:39 2022] GET /identity/v3/role_assignments?user.id=8b7abab7eef5492bbe7e7b0426c8971c&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=15be693919a24ac69f0c9320bba5fac2 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:40.626414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0c45-a85e-49ff-aac4-7894b8e4a680 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:40.626914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0c45-a85e-49ff-aac4-7894b8e4a680 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:40.627085 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e0c45-a85e-49ff-aac4-7894b8e4a680 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:40.628459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 151/319] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:40.633661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c6857-0d5a-47df-a6dc-0f0bc6997b0d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:40.634055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c6857-0d5a-47df-a6dc-0f0bc6997b0d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:40.634243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992c6857-0d5a-47df-a6dc-0f0bc6997b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:40.653403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-992c6857-0d5a-47df-a6dc-0f0bc6997b0d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:40.654632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-992c6857-0d5a-47df-a6dc-0f0bc6997b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:40.671619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4d00d-2735-4ae5-9a0f-538b745b215e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:40.671929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4d00d-2735-4ae5-9a0f-538b745b215e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:40.672207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa4d00d-2735-4ae5-9a0f-538b745b215e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:40.673537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 152/320] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:40.679075 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973a713-8e63-4783-9ae9-47848f64cf47 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:40.679322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973a713-8e63-4783-9ae9-47848f64cf47 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:40.679624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973a713-8e63-4783-9ae9-47848f64cf47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:40.681519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:40.699889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b973a713-8e63-4783-9ae9-47848f64cf47 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:40.701343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b973a713-8e63-4783-9ae9-47848f64cf47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:40.728461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 169/321] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:40.729083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:40.771890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 153/322] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:40.994535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c6857-0d5a-47df-a6dc-0f0bc6997b0d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:41.033510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973a713-8e63-4783-9ae9-47848f64cf47 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:41.099435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992c6857-0d5a-47df-a6dc-0f0bc6997b0d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:41.134550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed73dc-f4fd-49d9-a747-ae31782a333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:41.135189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed73dc-f4fd-49d9-a747-ae31782a333b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:41.135472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed73dc-f4fd-49d9-a747-ae31782a333b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:41.135743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ed73dc-f4fd-49d9-a747-ae31782a333b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:41.136847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed73dc-f4fd-49d9-a747-ae31782a333b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:41.138417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ed73dc-f4fd-49d9-a747-ae31782a333b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:41.139287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973a713-8e63-4783-9ae9-47848f64cf47 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:41.140260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 170/323] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:47:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:41.145502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed73dc-f4fd-49d9-a747-ae31782a333b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:41.174988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:41.179842 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6db79b-4b37-4958-8a18-a3942aeebb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:41.180487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6db79b-4b37-4958-8a18-a3942aeebb01 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:41.180760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6db79b-4b37-4958-8a18-a3942aeebb01 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:41.181067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6db79b-4b37-4958-8a18-a3942aeebb01 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:41.192216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6db79b-4b37-4958-8a18-a3942aeebb01 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-de6db79b-4b37-4958-8a18-a3942aeebb01 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:41.197087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:41.197972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:41.199156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 154/324] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:41.204325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6db79b-4b37-4958-8a18-a3942aeebb01 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:41.229550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:41.256491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ed73dc-f4fd-49d9-a747-ae31782a333b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:41.292042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c376e1-d9a3-4ab9-b05a-6a561b3f2eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:41.292658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c376e1-d9a3-4ab9-b05a-6a561b3f2eac None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:41.292937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c376e1-d9a3-4ab9-b05a-6a561b3f2eac None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:41.293228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c376e1-d9a3-4ab9-b05a-6a561b3f2eac None admin] PATH_INFO: `/v3/groups` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:41.293841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c376e1-d9a3-4ab9-b05a-6a561b3f2eac None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:41.295515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c376e1-d9a3-4ab9-b05a-6a561b3f2eac None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:41.312341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6db79b-4b37-4958-8a18-a3942aeebb01 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:41.313934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 171/325] 10.222.0.25 () {60 vars in 1134 bytes} [Tue Sep 13 06:47:41 2022] POST /identity/v3/groups => generated 226 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:47:41.355715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a87338-e285-4cc3-8073-04918b9b62db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:41.356352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a87338-e285-4cc3-8073-04918b9b62db None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:41.356858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a87338-e285-4cc3-8073-04918b9b62db None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:41.356858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a87338-e285-4cc3-8073-04918b9b62db None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:41.357744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a87338-e285-4cc3-8073-04918b9b62db None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:41.358066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a87338-e285-4cc3-8073-04918b9b62db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:41.359528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a87338-e285-4cc3-8073-04918b9b62db None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:41.369610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 155/326] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:41.375720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c376e1-d9a3-4ab9-b05a-6a561b3f2eac None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:41.480751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:41.502190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c376e1-d9a3-4ab9-b05a-6a561b3f2eac None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:41.536222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee02f73-38a0-4799-afd4-8426981d0229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:41.536789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee02f73-38a0-4799-afd4-8426981d0229 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:41.537067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee02f73-38a0-4799-afd4-8426981d0229 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:41.537362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee02f73-38a0-4799-afd4-8426981d0229 None admin] PATH_INFO: `/v3/users/1ccd550aa2cb4504862efd42b8104f1c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:41.538564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee02f73-38a0-4799-afd4-8426981d0229 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccd550aa2cb4504862efd42b8104f1c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:41.540024 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee02f73-38a0-4799-afd4-8426981d0229 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:41.541619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 172/327] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:41 2022] GET /identity/v3/users/1ccd550aa2cb4504862efd42b8104f1c => generated 293 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:41.544623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e7c8f-2057-44c5-b19d-792a3a61f656 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:41.544900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e7c8f-2057-44c5-b19d-792a3a61f656 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:41.545170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e7c8f-2057-44c5-b19d-792a3a61f656 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:41.546712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 156/328] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:41.547732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee02f73-38a0-4799-afd4-8426981d0229 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:41.555816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f71abe-b066-4fd5-bbb7-49fd816bc091 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:41.556278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f71abe-b066-4fd5-bbb7-49fd816bc091 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:41.556718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f71abe-b066-4fd5-bbb7-49fd816bc091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:41.579351 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-25f71abe-b066-4fd5-bbb7-49fd816bc091 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:41.580940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-25f71abe-b066-4fd5-bbb7-49fd816bc091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:41.671717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee02f73-38a0-4799-afd4-8426981d0229 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:41.674688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 157/329] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:41.709051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2890d-a843-48cc-ba86-36a626e0d32e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:41.709717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2890d-a843-48cc-ba86-36a626e0d32e None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:41.709993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2890d-a843-48cc-ba86-36a626e0d32e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:41.710291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b2890d-a843-48cc-ba86-36a626e0d32e None admin] PATH_INFO: `/v3/system/users/1ccd550aa2cb4504862efd42b8104f1c/roles/15be693919a24ac69f0c9320bba5fac2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:41.711714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2890d-a843-48cc-ba86-36a626e0d32e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1ccd550aa2cb4504862efd42b8104f1c, role_id=15be693919a24ac69f0c9320bba5fac2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:41.713108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b2890d-a843-48cc-ba86-36a626e0d32e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:41.722917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 173/330] 10.222.0.25 () {58 vars in 1334 bytes} [Tue Sep 13 06:47:41 2022] PUT /identity/v3/system/users/1ccd550aa2cb4504862efd42b8104f1c/roles/15be693919a24ac69f0c9320bba5fac2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:41.945931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f71abe-b066-4fd5-bbb7-49fd816bc091 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:41.962768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:42.061929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f71abe-b066-4fd5-bbb7-49fd816bc091 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:42.096508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56004c54-d722-4e86-a7ec-38f44c77215b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:42.097144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56004c54-d722-4e86-a7ec-38f44c77215b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:42.097418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56004c54-d722-4e86-a7ec-38f44c77215b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:42.097686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56004c54-d722-4e86-a7ec-38f44c77215b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:42.110291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56004c54-d722-4e86-a7ec-38f44c77215b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-56004c54-d722-4e86-a7ec-38f44c77215b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:42.115259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:42.116112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:42.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 158/331] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:42.127264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2890d-a843-48cc-ba86-36a626e0d32e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:42.159515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:42.258718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b2890d-a843-48cc-ba86-36a626e0d32e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:42.299648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336bed0-bed1-4cc3-87bf-df9f9b869740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:42.300284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336bed0-bed1-4cc3-87bf-df9f9b869740 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:42.300559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336bed0-bed1-4cc3-87bf-df9f9b869740 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:42.300828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336bed0-bed1-4cc3-87bf-df9f9b869740 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:42.301507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336bed0-bed1-4cc3-87bf-df9f9b869740 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:42.301807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336bed0-bed1-4cc3-87bf-df9f9b869740 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:42.303233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336bed0-bed1-4cc3-87bf-df9f9b869740 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:42.313821 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 174/332] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:42.320445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56004c54-d722-4e86-a7ec-38f44c77215b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:42.422636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:42.451987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56004c54-d722-4e86-a7ec-38f44c77215b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:42.486838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da320b5-680b-4b77-9684-ef20656d9a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:42.487487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da320b5-680b-4b77-9684-ef20656d9a3d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:42.487769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da320b5-680b-4b77-9684-ef20656d9a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:42.488040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da320b5-680b-4b77-9684-ef20656d9a3d None admin] PATH_INFO: `/v3/users/55ad0ae539644ddcb274eb6ccfb3c4aa` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:42.489190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da320b5-680b-4b77-9684-ef20656d9a3d None admin] RBAC: Authorizing `identity:get_user(user_id=55ad0ae539644ddcb274eb6ccfb3c4aa)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:42.490683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da320b5-680b-4b77-9684-ef20656d9a3d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:42.492646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 159/333] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:42 2022] GET /identity/v3/users/55ad0ae539644ddcb274eb6ccfb3c4aa => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:42.499068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336bed0-bed1-4cc3-87bf-df9f9b869740 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:42.631728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:42.654291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336bed0-bed1-4cc3-87bf-df9f9b869740 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:42.698102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453061-e24d-470e-9904-89cb88f90d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:42.698752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453061-e24d-470e-9904-89cb88f90d22 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:42.699015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453061-e24d-470e-9904-89cb88f90d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:42.699292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453061-e24d-470e-9904-89cb88f90d22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:42.700027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453061-e24d-470e-9904-89cb88f90d22 None admin] RBAC: Adding query filter params (role.id=2d59fcad66104464bbc848ef605f743e, user.id=55ad0ae539644ddcb274eb6ccfb3c4aa, scope.system=all) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:42.700489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453061-e24d-470e-9904-89cb88f90d22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:42.702105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453061-e24d-470e-9904-89cb88f90d22 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:42.717232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 175/334] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Sep 13 06:47:42 2022] GET /identity/v3/role_assignments?user.id=55ad0ae539644ddcb274eb6ccfb3c4aa&scope.system=all&role.id=2d59fcad66104464bbc848ef605f743e => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:44.032441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba69f8-1e05-448d-9e2e-827682e4f6a2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:44.033443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba69f8-1e05-448d-9e2e-827682e4f6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:44.033981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba69f8-1e05-448d-9e2e-827682e4f6a2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:44.035571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 160/335] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:44.041202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45d33d-e58d-46de-8826-29862a792b99 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:44.041676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45d33d-e58d-46de-8826-29862a792b99 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:44.041762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e45d33d-e58d-46de-8826-29862a792b99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:44.067128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0e45d33d-e58d-46de-8826-29862a792b99 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:44.068343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0e45d33d-e58d-46de-8826-29862a792b99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:44.141708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 176/336] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:44.579223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba69f8-1e05-448d-9e2e-827682e4f6a2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:44.718480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba69f8-1e05-448d-9e2e-827682e4f6a2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:44.770125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c8336-6fca-400e-ad49-45fb9ae7843d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:44.770793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c8336-6fca-400e-ad49-45fb9ae7843d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:44.771071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c8336-6fca-400e-ad49-45fb9ae7843d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:44.771345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446c8336-6fca-400e-ad49-45fb9ae7843d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:44.786739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446c8336-6fca-400e-ad49-45fb9ae7843d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:44.787349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74321f-dbb6-46e8-9e9c-cdcecd0778dd None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:44.787631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74321f-dbb6-46e8-9e9c-cdcecd0778dd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:44.787901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a74321f-dbb6-46e8-9e9c-cdcecd0778dd None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:44.789494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 177/337] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-446c8336-6fca-400e-ad49-45fb9ae7843d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:44.791795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:44.792736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:44.794045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 161/338] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:44.795098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaefde9-05b7-4476-bfd5-520ef71c8254 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:44.795181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaefde9-05b7-4476-bfd5-520ef71c8254 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:44.795412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eaefde9-05b7-4476-bfd5-520ef71c8254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:44.799742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c8336-6fca-400e-ad49-45fb9ae7843d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:44.806384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:44.839224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5eaefde9-05b7-4476-bfd5-520ef71c8254 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:44.840332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5eaefde9-05b7-4476-bfd5-520ef71c8254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:44.942781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:44.952355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 178/339] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:47:44.954130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420de66-0b47-41b2-aefc-962e1eda8173 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:44.954723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420de66-0b47-41b2-aefc-962e1eda8173 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:44.955221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f420de66-0b47-41b2-aefc-962e1eda8173 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:44.957023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 179/340] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:44.958460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b1c9c-12bc-40c9-92d8-0e7d6562091f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:44.958950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b1c9c-12bc-40c9-92d8-0e7d6562091f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:44.959425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359b1c9c-12bc-40c9-92d8-0e7d6562091f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:44.960489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446c8336-6fca-400e-ad49-45fb9ae7843d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:44.961114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 180/341] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:44.966567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc62fc-72b0-4856-bfc7-553a7623b2eb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:44.967100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc62fc-72b0-4856-bfc7-553a7623b2eb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:44.967580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cc62fc-72b0-4856-bfc7-553a7623b2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:44.987394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d3cc62fc-72b0-4856-bfc7-553a7623b2eb None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:44.989042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3cc62fc-72b0-4856-bfc7-553a7623b2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:44.998770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c369cd4-601e-4741-94e8-936f945486a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:44.999395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c369cd4-601e-4741-94e8-936f945486a7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:44.999674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c369cd4-601e-4741-94e8-936f945486a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:44.999944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c369cd4-601e-4741-94e8-936f945486a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:45.000622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c369cd4-601e-4741-94e8-936f945486a7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:45.000924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c369cd4-601e-4741-94e8-936f945486a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:45.007042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c369cd4-601e-4741-94e8-936f945486a7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:45.011294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 162/342] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:45.012541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f1ed2-01af-49eb-a7cc-315f5ed95832 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:45.012885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f1ed2-01af-49eb-a7cc-315f5ed95832 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:45.013182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f1ed2-01af-49eb-a7cc-315f5ed95832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:45.033980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-060f1ed2-01af-49eb-a7cc-315f5ed95832 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:45.035222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-060f1ed2-01af-49eb-a7cc-315f5ed95832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:45.098554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:45.100548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 181/343] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:45.103072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc62fc-72b0-4856-bfc7-553a7623b2eb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:45.114169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 163/344] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:45.224864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cc62fc-72b0-4856-bfc7-553a7623b2eb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:45.266811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:45.268460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f1ed2-01af-49eb-a7cc-315f5ed95832 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:45.273407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0e8fd-73e7-4fc2-a5fa-8c72db00f5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:45.274113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0e8fd-73e7-4fc2-a5fa-8c72db00f5e1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:45.274402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0e8fd-73e7-4fc2-a5fa-8c72db00f5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:45.274674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b0e8fd-73e7-4fc2-a5fa-8c72db00f5e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:45.389542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f1ed2-01af-49eb-a7cc-315f5ed95832 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:45.421983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0e8fd-73e7-4fc2-a5fa-8c72db00f5e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:45.423509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b0e8fd-73e7-4fc2-a5fa-8c72db00f5e1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:45.465016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ded1d-87bf-4eb2-afa2-e046c17ec1ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:45.465696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ded1d-87bf-4eb2-afa2-e046c17ec1ea None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:45.466020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ded1d-87bf-4eb2-afa2-e046c17ec1ea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:45.467424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ded1d-87bf-4eb2-afa2-e046c17ec1ea None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:45.501159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe3ded1d-87bf-4eb2-afa2-e046c17ec1ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fe3ded1d-87bf-4eb2-afa2-e046c17ec1ea None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:45.511763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:45.512817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:45.514415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:45.517734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 164/345] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:45 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:47:45.519020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ded1d-87bf-4eb2-afa2-e046c17ec1ea None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:45.604005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 182/346] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:45.604708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0e8fd-73e7-4fc2-a5fa-8c72db00f5e1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:45.618144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:45.662305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:45.670825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe3ded1d-87bf-4eb2-afa2-e046c17ec1ea None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:45.710587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78710e-4d58-4d9a-9ba8-75c29b538c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:45.711215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78710e-4d58-4d9a-9ba8-75c29b538c91 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:45.711493 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78710e-4d58-4d9a-9ba8-75c29b538c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:45.711759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e78710e-4d58-4d9a-9ba8-75c29b538c91 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:45.712716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78710e-4d58-4d9a-9ba8-75c29b538c91 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:45.713013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78710e-4d58-4d9a-9ba8-75c29b538c91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:45.715773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e78710e-4d58-4d9a-9ba8-75c29b538c91 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:45.724987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 165/347] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:45.725969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78710e-4d58-4d9a-9ba8-75c29b538c91 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:45.751010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b0e8fd-73e7-4fc2-a5fa-8c72db00f5e1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:45.794646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9311e-379e-43c1-a6d7-f1e1ddf3b67b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:45.795722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9311e-379e-43c1-a6d7-f1e1ddf3b67b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:45.796154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9311e-379e-43c1-a6d7-f1e1ddf3b67b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:45.796566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d9311e-379e-43c1-a6d7-f1e1ddf3b67b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:45.811875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d9311e-379e-43c1-a6d7-f1e1ddf3b67b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-22d9311e-379e-43c1-a6d7-f1e1ddf3b67b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:45.818473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:45.819751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:45.820720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:45.824155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 183/348] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:45.825506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9311e-379e-43c1-a6d7-f1e1ddf3b67b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:45.883512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e78710e-4d58-4d9a-9ba8-75c29b538c91 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:45.902003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:45.928498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070b395-70de-4936-882a-4b811306ab68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:45.929313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070b395-70de-4936-882a-4b811306ab68 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:45.929713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070b395-70de-4936-882a-4b811306ab68 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:45.930112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0070b395-70de-4936-882a-4b811306ab68 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:45.931483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070b395-70de-4936-882a-4b811306ab68 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:45.933476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0070b395-70de-4936-882a-4b811306ab68 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:45.936770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 166/349] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:45 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:45.937771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070b395-70de-4936-882a-4b811306ab68 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:45.945643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:45.960370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d9311e-379e-43c1-a6d7-f1e1ddf3b67b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:46.004566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ee2d4-9f3c-4694-b0bd-bfaf43b2b3a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:46.004812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ee2d4-9f3c-4694-b0bd-bfaf43b2b3a7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.005267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ee2d4-9f3c-4694-b0bd-bfaf43b2b3a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.005678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62ee2d4-9f3c-4694-b0bd-bfaf43b2b3a7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.020212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62ee2d4-9f3c-4694-b0bd-bfaf43b2b3a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a62ee2d4-9f3c-4694-b0bd-bfaf43b2b3a7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:46.026286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:46.027223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 13 06:47:46.028013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:46.028908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 184/350] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:46.029998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ee2d4-9f3c-4694-b0bd-bfaf43b2b3a7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:46.068532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0070b395-70de-4936-882a-4b811306ab68 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:46.118706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d61b658-97b9-4d8a-be90-7fa4479c2d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:46.119373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61b658-97b9-4d8a-be90-7fa4479c2d4b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.119664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61b658-97b9-4d8a-be90-7fa4479c2d4b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.119918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d61b658-97b9-4d8a-be90-7fa4479c2d4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.160245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62ee2d4-9f3c-4694-b0bd-bfaf43b2b3a7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:46.184071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:46.205339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adae328-11ba-47fd-ac37-555dd1c515fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:46.205967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adae328-11ba-47fd-ac37-555dd1c515fa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.206303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adae328-11ba-47fd-ac37-555dd1c515fa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.207110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adae328-11ba-47fd-ac37-555dd1c515fa None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.207795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adae328-11ba-47fd-ac37-555dd1c515fa None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:46.208252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adae328-11ba-47fd-ac37-555dd1c515fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:46.208848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61b658-97b9-4d8a-be90-7fa4479c2d4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:46.209678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adae328-11ba-47fd-ac37-555dd1c515fa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:46.210360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d61b658-97b9-4d8a-be90-7fa4479c2d4b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:46.220032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 185/351] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:46.220899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adae328-11ba-47fd-ac37-555dd1c515fa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:46.234762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:46.340770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 167/352] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:46.343263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120dbbc-464e-423a-9766-c69345995821 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.343526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120dbbc-464e-423a-9766-c69345995821 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.343803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120dbbc-464e-423a-9766-c69345995821 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.345243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 168/353] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:46.345883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120dbbc-464e-423a-9766-c69345995821 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:46.355752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adae328-11ba-47fd-ac37-555dd1c515fa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:46.397540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cde1d4-b756-410b-a00c-59277e8aa252 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:46.398222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cde1d4-b756-410b-a00c-59277e8aa252 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.398505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cde1d4-b756-410b-a00c-59277e8aa252 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.398784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cde1d4-b756-410b-a00c-59277e8aa252 None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.406297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cde1d4-b756-410b-a00c-59277e8aa252 None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:46.407780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82cde1d4-b756-410b-a00c-59277e8aa252 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:46.409483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 186/354] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:46 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:46.410964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cde1d4-b756-410b-a00c-59277e8aa252 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:46.454443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:46.477557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120dbbc-464e-423a-9766-c69345995821 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:46.503679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:46.523861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463f4ce1-f350-4b81-b063-e976c1ce6b29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:46.524487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f4ce1-f350-4b81-b063-e976c1ce6b29 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.524768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f4ce1-f350-4b81-b063-e976c1ce6b29 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.525135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463f4ce1-f350-4b81-b063-e976c1ce6b29 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.525708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f4ce1-f350-4b81-b063-e976c1ce6b29 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:46.526017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f4ce1-f350-4b81-b063-e976c1ce6b29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:46.527468 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463f4ce1-f350-4b81-b063-e976c1ce6b29 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:46.537214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 169/355] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:46.538637 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31775f58-95ab-4ab7-96e1-eb4326351c94 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.538931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31775f58-95ab-4ab7-96e1-eb4326351c94 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.539210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31775f58-95ab-4ab7-96e1-eb4326351c94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.539587 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82cde1d4-b756-410b-a00c-59277e8aa252 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:46.559297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-31775f58-95ab-4ab7-96e1-eb4326351c94 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:46.560724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31775f58-95ab-4ab7-96e1-eb4326351c94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:46.583036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df411e-f44a-40d8-bfc6-8bcb7c7210a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:46.583332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df411e-f44a-40d8-bfc6-8bcb7c7210a3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.583615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df411e-f44a-40d8-bfc6-8bcb7c7210a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.583890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1df411e-f44a-40d8-bfc6-8bcb7c7210a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.642654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 170/356] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:46.643846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31775f58-95ab-4ab7-96e1-eb4326351c94 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:46.682183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df411e-f44a-40d8-bfc6-8bcb7c7210a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:46.684464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1df411e-f44a-40d8-bfc6-8bcb7c7210a3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:46.738642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:46.766631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31775f58-95ab-4ab7-96e1-eb4326351c94 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:46.777749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:46.813890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 187/357] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:46.814570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7221185-14f4-4db8-8c94-ce7e0be6f78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:46.814868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df411e-f44a-40d8-bfc6-8bcb7c7210a3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:46.815744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7221185-14f4-4db8-8c94-ce7e0be6f78d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.816229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7221185-14f4-4db8-8c94-ce7e0be6f78d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.816724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7221185-14f4-4db8-8c94-ce7e0be6f78d None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.818410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7221185-14f4-4db8-8c94-ce7e0be6f78d None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:46.820152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7221185-14f4-4db8-8c94-ce7e0be6f78d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:46.822596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 171/358] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:46 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:46.823873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7221185-14f4-4db8-8c94-ce7e0be6f78d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:46.927562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1df411e-f44a-40d8-bfc6-8bcb7c7210a3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:46.936125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7221185-14f4-4db8-8c94-ce7e0be6f78d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:46.968104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fd720-c963-4a84-956a-8fac7264d028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:46.968728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd720-c963-4a84-956a-8fac7264d028 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.969006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd720-c963-4a84-956a-8fac7264d028 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.969282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd720-c963-4a84-956a-8fac7264d028 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.970027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd720-c963-4a84-956a-8fac7264d028 None admin] RBAC: Adding query filter params (group.id=c780b10d545b439da0b83c42a939a617, role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=ab8379e77b244774a8bf8329866e9014) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:46.970354 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd720-c963-4a84-956a-8fac7264d028 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:46.971812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd720-c963-4a84-956a-8fac7264d028 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:46.974515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1c73c-d79f-474d-a1cb-af4e20897b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:46.975157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1c73c-d79f-474d-a1cb-af4e20897b66 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:46.975436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1c73c-d79f-474d-a1cb-af4e20897b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:46.975707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d1c73c-d79f-474d-a1cb-af4e20897b66 None admin] PATH_INFO: `/v3/users/1ccd550aa2cb4504862efd42b8104f1c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:46.976838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1c73c-d79f-474d-a1cb-af4e20897b66 None admin] RBAC: Authorizing `identity:get_user(user_id=1ccd550aa2cb4504862efd42b8104f1c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:46.977572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 188/359] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:46 2022] GET /identity/v3/role_assignments?group.id=c780b10d545b439da0b83c42a939a617&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=2d59fcad66104464bbc848ef605f743e => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:46.978293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fd720-c963-4a84-956a-8fac7264d028 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:46.978412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d1c73c-d79f-474d-a1cb-af4e20897b66 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:46.979895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 172/360] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:47:46 2022] GET /identity/v3/users/1ccd550aa2cb4504862efd42b8104f1c => generated 293 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:46.980971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1c73c-d79f-474d-a1cb-af4e20897b66 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:47.002741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:47.032130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:47.112550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fd720-c963-4a84-956a-8fac7264d028 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:47.124458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d1c73c-d79f-474d-a1cb-af4e20897b66 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:47.154676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da799213-1dd2-4f8e-ad42-0a48fe2de67c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:47.155339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da799213-1dd2-4f8e-ad42-0a48fe2de67c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.155616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da799213-1dd2-4f8e-ad42-0a48fe2de67c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.155889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da799213-1dd2-4f8e-ad42-0a48fe2de67c None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.157077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da799213-1dd2-4f8e-ad42-0a48fe2de67c None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:47.158600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da799213-1dd2-4f8e-ad42-0a48fe2de67c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:47.160296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 189/361] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:46 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:47.161497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36ec2a-41b4-4e3b-a2e6-08344cc8d989 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.161836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36ec2a-41b4-4e3b-a2e6-08344cc8d989 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.162135 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a36ec2a-41b4-4e3b-a2e6-08344cc8d989 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.164350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 190/362] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:47.164350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36ec2a-41b4-4e3b-a2e6-08344cc8d989 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:47.168625 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e0b20-3a16-4990-a43e-b8ff19564fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:47.169550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e0b20-3a16-4990-a43e-b8ff19564fbd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.169725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e0b20-3a16-4990-a43e-b8ff19564fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.169950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433e0b20-3a16-4990-a43e-b8ff19564fbd None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.171260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e0b20-3a16-4990-a43e-b8ff19564fbd None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:47.172765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433e0b20-3a16-4990-a43e-b8ff19564fbd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:47.174445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 173/363] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:46 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:47.175313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e0b20-3a16-4990-a43e-b8ff19564fbd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:47.273382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:47.300430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a36ec2a-41b4-4e3b-a2e6-08344cc8d989 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:47.303781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433e0b20-3a16-4990-a43e-b8ff19564fbd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:47.309492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:47.342829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe123b9-3390-48d8-9941-b68b8f2256d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:47.343479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe123b9-3390-48d8-9941-b68b8f2256d9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.343757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe123b9-3390-48d8-9941-b68b8f2256d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.344031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe123b9-3390-48d8-9941-b68b8f2256d9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.344838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213128-a647-4d87-ac08-1e04c59a67d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:47.345464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213128-a647-4d87-ac08-1e04c59a67d2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.345739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213128-a647-4d87-ac08-1e04c59a67d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.346061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d213128-a647-4d87-ac08-1e04c59a67d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.347083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d213128-a647-4d87-ac08-1e04c59a67d2 None admin] RBAC: Adding query filter params (role.id=15be693919a24ac69f0c9320bba5fac2, user.id=1ccd550aa2cb4504862efd42b8104f1c, scope.system=all) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:47.347200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d213128-a647-4d87-ac08-1e04c59a67d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:47.348558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d213128-a647-4d87-ac08-1e04c59a67d2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:47.358532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe123b9-3390-48d8-9941-b68b8f2256d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:47.361681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 174/364] 10.222.0.25 () {56 vars in 1308 bytes} [Tue Sep 13 06:47:47 2022] GET /identity/v3/role_assignments?user.id=1ccd550aa2cb4504862efd42b8104f1c&scope.system=all&role.id=15be693919a24ac69f0c9320bba5fac2 => generated 510 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:47.364862 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213128-a647-4d87-ac08-1e04c59a67d2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-dfe123b9-3390-48d8-9941-b68b8f2256d9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:47.364993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:47.365865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 191/365] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:47.366720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75787277-5447-4824-8927-76d8b152f6d9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.367473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75787277-5447-4824-8927-76d8b152f6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.367473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75787277-5447-4824-8927-76d8b152f6d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.390849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-75787277-5447-4824-8927-76d8b152f6d9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:47.392206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-75787277-5447-4824-8927-76d8b152f6d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:47.477886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 192/366] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:47.480801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75787277-5447-4824-8927-76d8b152f6d9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:47.504059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d213128-a647-4d87-ac08-1e04c59a67d2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:47.548072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c5258-0763-41d5-8948-67cde9b475d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:47.548654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c5258-0763-41d5-8948-67cde9b475d4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.548933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c5258-0763-41d5-8948-67cde9b475d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.549206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393c5258-0763-41d5-8948-67cde9b475d4 None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.550599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c5258-0763-41d5-8948-67cde9b475d4 None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:47.552228 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393c5258-0763-41d5-8948-67cde9b475d4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:47.554536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 175/367] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:47 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:47.555355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c5258-0763-41d5-8948-67cde9b475d4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:47.567160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:47.634847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:47.641155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75787277-5447-4824-8927-76d8b152f6d9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:47.680847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6f852-9259-4381-86e6-8f925631461d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:47.681642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6f852-9259-4381-86e6-8f925631461d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.682019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6f852-9259-4381-86e6-8f925631461d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.682417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c6f852-9259-4381-86e6-8f925631461d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.683594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6f852-9259-4381-86e6-8f925631461d None admin] RBAC: Adding query filter params (group.id=c780b10d545b439da0b83c42a939a617, role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=ab8379e77b244774a8bf8329866e9014) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:47.684000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6f852-9259-4381-86e6-8f925631461d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:47.686386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c6f852-9259-4381-86e6-8f925631461d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:47.692492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393c5258-0763-41d5-8948-67cde9b475d4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:47.694754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 193/368] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:47 2022] GET /identity/v3/role_assignments?group.id=c780b10d545b439da0b83c42a939a617&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=c622170611454f18bf01aa658a8cc06e => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:47.695707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6f852-9259-4381-86e6-8f925631461d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:47.733471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f569122-5a79-478e-bf3a-2735ea319fc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:47.734179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f569122-5a79-478e-bf3a-2735ea319fc2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.734461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f569122-5a79-478e-bf3a-2735ea319fc2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.734735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f569122-5a79-478e-bf3a-2735ea319fc2 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.735419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f569122-5a79-478e-bf3a-2735ea319fc2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:47.735719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f569122-5a79-478e-bf3a-2735ea319fc2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:47.737177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f569122-5a79-478e-bf3a-2735ea319fc2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:47.748505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 176/369] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:47 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:47.758727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f569122-5a79-478e-bf3a-2735ea319fc2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:47.847922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:47.884388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c6f852-9259-4381-86e6-8f925631461d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:47.894526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f569122-5a79-478e-bf3a-2735ea319fc2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:47.924610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f325a11-dd4b-4d11-9fc2-88d30bd04a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:47.925340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f325a11-dd4b-4d11-9fc2-88d30bd04a10 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.925747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f325a11-dd4b-4d11-9fc2-88d30bd04a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.926144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f325a11-dd4b-4d11-9fc2-88d30bd04a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.926973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f325a11-dd4b-4d11-9fc2-88d30bd04a10 None admin] RBAC: Adding query filter params (group.id=c780b10d545b439da0b83c42a939a617, role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=fe3324999301468c8543ebc979dbd819) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:47.927353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f325a11-dd4b-4d11-9fc2-88d30bd04a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:47.928934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f325a11-dd4b-4d11-9fc2-88d30bd04a10 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:47.938644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 194/370] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:47 2022] GET /identity/v3/role_assignments?group.id=c780b10d545b439da0b83c42a939a617&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=2d59fcad66104464bbc848ef605f743e => generated 260 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:47.943113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05845e24-34a8-4057-a9ee-2c8ea6838295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:47.943896 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05845e24-34a8-4057-a9ee-2c8ea6838295 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:47.944277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05845e24-34a8-4057-a9ee-2c8ea6838295 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:47.944648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05845e24-34a8-4057-a9ee-2c8ea6838295 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:47.958598 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f325a11-dd4b-4d11-9fc2-88d30bd04a10 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:48.001729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:48.061453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05845e24-34a8-4057-a9ee-2c8ea6838295 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:48.063283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05845e24-34a8-4057-a9ee-2c8ea6838295 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:48.163832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f325a11-dd4b-4d11-9fc2-88d30bd04a10 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:48.174996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:48.214978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 177/371] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:48.221682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05845e24-34a8-4057-a9ee-2c8ea6838295 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:48.222574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85978d-6bc8-41e7-96a2-f505da1f160a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:48.223240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85978d-6bc8-41e7-96a2-f505da1f160a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:48.223530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85978d-6bc8-41e7-96a2-f505da1f160a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:48.224029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a85978d-6bc8-41e7-96a2-f505da1f160a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:48.242069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a85978d-6bc8-41e7-96a2-f505da1f160a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7a85978d-6bc8-41e7-96a2-f505da1f160a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:48.249013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:48.251332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:48.252864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 195/372] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:47:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:47:48.260198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85978d-6bc8-41e7-96a2-f505da1f160a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:48.356360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:48.374480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a85978d-6bc8-41e7-96a2-f505da1f160a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:48.410255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a124f-88ce-4408-b318-312508a15ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:48.410876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a124f-88ce-4408-b318-312508a15ee6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:48.411150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a124f-88ce-4408-b318-312508a15ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:48.411417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a124f-88ce-4408-b318-312508a15ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:48.412085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a124f-88ce-4408-b318-312508a15ee6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:48.412382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a124f-88ce-4408-b318-312508a15ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:48.413779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70a124f-88ce-4408-b318-312508a15ee6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:48.423573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 196/373] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:48.429123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a124f-88ce-4408-b318-312508a15ee6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:48.446196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05845e24-34a8-4057-a9ee-2c8ea6838295 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:48.508438 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e401c4-afc7-4b86-90c5-1c3d956d6e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:48.509102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e401c4-afc7-4b86-90c5-1c3d956d6e60 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:48.509380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e401c4-afc7-4b86-90c5-1c3d956d6e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:48.509651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e401c4-afc7-4b86-90c5-1c3d956d6e60 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:48.514823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e401c4-afc7-4b86-90c5-1c3d956d6e60 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:48.516300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e401c4-afc7-4b86-90c5-1c3d956d6e60 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:48.518413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 178/374] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:48 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:48.523623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e401c4-afc7-4b86-90c5-1c3d956d6e60 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:48.566681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70a124f-88ce-4408-b318-312508a15ee6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:48.574515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:48.604317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b48312-5497-4809-82fe-9fdcc7804b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:48.605215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b48312-5497-4809-82fe-9fdcc7804b19 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:48.605497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b48312-5497-4809-82fe-9fdcc7804b19 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:48.605917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b48312-5497-4809-82fe-9fdcc7804b19 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:48.622961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:48.680954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b48312-5497-4809-82fe-9fdcc7804b19 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:48.686101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b48312-5497-4809-82fe-9fdcc7804b19 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:48.716266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e401c4-afc7-4b86-90c5-1c3d956d6e60 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:48.787848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 197/375] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:48.793680 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b48312-5497-4809-82fe-9fdcc7804b19 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:48.805012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f6cc-5a17-4795-afc9-daab09c07bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:48.805664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f6cc-5a17-4795-afc9-daab09c07bab None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:48.805941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f6cc-5a17-4795-afc9-daab09c07bab None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:48.806237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0f6cc-5a17-4795-afc9-daab09c07bab None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:48.808763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0f6cc-5a17-4795-afc9-daab09c07bab None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:48.813647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0f6cc-5a17-4795-afc9-daab09c07bab None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:48.815762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 179/376] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:48 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:48.822664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f6cc-5a17-4795-afc9-daab09c07bab None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:48.883422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:48.992234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b48312-5497-4809-82fe-9fdcc7804b19 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:49.008117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0f6cc-5a17-4795-afc9-daab09c07bab None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:49.010507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:49.032634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a9dfc-b65c-4ba7-9c95-e27c72aa6f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:49.033281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a9dfc-b65c-4ba7-9c95-e27c72aa6f54 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:49.033556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a9dfc-b65c-4ba7-9c95-e27c72aa6f54 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:49.033827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315a9dfc-b65c-4ba7-9c95-e27c72aa6f54 None admin] PATH_INFO: `/v3/projects/2becfff2f1104b709c0a7babcac266dd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:49.035035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a9dfc-b65c-4ba7-9c95-e27c72aa6f54 None admin] RBAC: Authorizing `identity:get_project(project_id=2becfff2f1104b709c0a7babcac266dd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:49.036606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315a9dfc-b65c-4ba7-9c95-e27c72aa6f54 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:49.038571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 198/377] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:48 2022] GET /identity/v3/projects/2becfff2f1104b709c0a7babcac266dd => generated 344 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:49.044180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a9dfc-b65c-4ba7-9c95-e27c72aa6f54 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:49.086869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9d7ff-ee1c-4e50-b599-754c05b195eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:49.087525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9d7ff-ee1c-4e50-b599-754c05b195eb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:49.087804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9d7ff-ee1c-4e50-b599-754c05b195eb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:49.088081 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9d7ff-ee1c-4e50-b599-754c05b195eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:49.088857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9d7ff-ee1c-4e50-b599-754c05b195eb None admin] RBAC: Adding query filter params (group.id=c780b10d545b439da0b83c42a939a617, role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=fe3324999301468c8543ebc979dbd819) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:49.089172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9d7ff-ee1c-4e50-b599-754c05b195eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:49.090740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9d7ff-ee1c-4e50-b599-754c05b195eb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:49.104891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 180/378] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:48 2022] GET /identity/v3/role_assignments?group.id=c780b10d545b439da0b83c42a939a617&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=c622170611454f18bf01aa658a8cc06e => generated 260 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:49.166217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315a9dfc-b65c-4ba7-9c95-e27c72aa6f54 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:49.182032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:49.205046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d824c5de-e713-4a76-993f-100685e1ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:49.205747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824c5de-e713-4a76-993f-100685e1ba91 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:49.206023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824c5de-e713-4a76-993f-100685e1ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:49.206315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d824c5de-e713-4a76-993f-100685e1ba91 None admin] PATH_INFO: `/v3/groups/504b2be6ba094c699f515c6e99cdbd40` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:49.211301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824c5de-e713-4a76-993f-100685e1ba91 None admin] RBAC: Authorizing `identity:get_group(group_id=504b2be6ba094c699f515c6e99cdbd40)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:49.212782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d824c5de-e713-4a76-993f-100685e1ba91 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:49.214841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 199/379] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:49 2022] GET /identity/v3/groups/504b2be6ba094c699f515c6e99cdbd40 => generated 229 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:49.220898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9d7ff-ee1c-4e50-b599-754c05b195eb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:49.364436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9d7ff-ee1c-4e50-b599-754c05b195eb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:49.398958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f3fca2-efb0-4e1f-a027-ee476e7c2239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:49.399573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3fca2-efb0-4e1f-a027-ee476e7c2239 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:49.399853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3fca2-efb0-4e1f-a027-ee476e7c2239 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:49.400123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f3fca2-efb0-4e1f-a027-ee476e7c2239 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:49.400850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3fca2-efb0-4e1f-a027-ee476e7c2239 None admin] RBAC: Adding query filter params (group.id=504b2be6ba094c699f515c6e99cdbd40, role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=2becfff2f1104b709c0a7babcac266dd) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:49.401157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3fca2-efb0-4e1f-a027-ee476e7c2239 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:49.402645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f3fca2-efb0-4e1f-a027-ee476e7c2239 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:49.408858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 181/380] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:49 2022] GET /identity/v3/role_assignments?group.id=504b2be6ba094c699f515c6e99cdbd40&scope.project.id=2becfff2f1104b709c0a7babcac266dd&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 260 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:49.763425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160c5aa-1ead-4534-971e-b520ca64aa42 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:49.763425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160c5aa-1ead-4534-971e-b520ca64aa42 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:49.763425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1160c5aa-1ead-4534-971e-b520ca64aa42 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:49.764438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 200/381] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:49.769877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078643bd-040d-46c0-b372-1ff9e6a40e48 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:49.770178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078643bd-040d-46c0-b372-1ff9e6a40e48 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:49.770453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078643bd-040d-46c0-b372-1ff9e6a40e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:49.799124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-078643bd-040d-46c0-b372-1ff9e6a40e48 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:49.800335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-078643bd-040d-46c0-b372-1ff9e6a40e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:49.832015 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:49.887754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 182/382] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:50.177987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160c5aa-1ead-4534-971e-b520ca64aa42 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:50.304960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1160c5aa-1ead-4534-971e-b520ca64aa42 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:50.349597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8def81b-95f7-4a11-a355-0ccaf5f74335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:50.350275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8def81b-95f7-4a11-a355-0ccaf5f74335 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:50.350549 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8def81b-95f7-4a11-a355-0ccaf5f74335 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:50.350815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8def81b-95f7-4a11-a355-0ccaf5f74335 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:50.362495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8def81b-95f7-4a11-a355-0ccaf5f74335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e8def81b-95f7-4a11-a355-0ccaf5f74335 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:50.371220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:50.373357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:50.375382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 201/383] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:50.382924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078643bd-040d-46c0-b372-1ff9e6a40e48 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:50.501193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078643bd-040d-46c0-b372-1ff9e6a40e48 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:50.546020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf768356-0ffb-4dea-aa67-60e1137df734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:50.546670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf768356-0ffb-4dea-aa67-60e1137df734 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:50.546945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf768356-0ffb-4dea-aa67-60e1137df734 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:50.547215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf768356-0ffb-4dea-aa67-60e1137df734 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:50.547886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf768356-0ffb-4dea-aa67-60e1137df734 None admin] RBAC: Adding query filter params (name=member) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:50.548195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf768356-0ffb-4dea-aa67-60e1137df734 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:50.549564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf768356-0ffb-4dea-aa67-60e1137df734 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:50.562809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 183/384] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:50.568366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8def81b-95f7-4a11-a355-0ccaf5f74335 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:50.614462 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:50.689844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8def81b-95f7-4a11-a355-0ccaf5f74335 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:50.733606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de00e29-258a-4594-a502-294f5d5e7947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:50.734296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de00e29-258a-4594-a502-294f5d5e7947 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:50.734571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de00e29-258a-4594-a502-294f5d5e7947 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:50.734842 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de00e29-258a-4594-a502-294f5d5e7947 None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:50.735963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de00e29-258a-4594-a502-294f5d5e7947 None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:50.737424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de00e29-258a-4594-a502-294f5d5e7947 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:50.739031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 202/385] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:50 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:50.745014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf768356-0ffb-4dea-aa67-60e1137df734 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:50.773879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:50.856975 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf768356-0ffb-4dea-aa67-60e1137df734 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:50.896192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed4ca2-aec9-4baf-a46c-e652d3b39552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:50.896854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed4ca2-aec9-4baf-a46c-e652d3b39552 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:50.897134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed4ca2-aec9-4baf-a46c-e652d3b39552 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:50.897407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed4ca2-aec9-4baf-a46c-e652d3b39552 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:50.898609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed4ca2-aec9-4baf-a46c-e652d3b39552 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:50.900080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed4ca2-aec9-4baf-a46c-e652d3b39552 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:50.902165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 184/386] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:50 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:50.908966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de00e29-258a-4594-a502-294f5d5e7947 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:51.030671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3840ae-0e48-405a-acae-6bafca6a02a0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:51.030968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3840ae-0e48-405a-acae-6bafca6a02a0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:51.031141 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3840ae-0e48-405a-acae-6bafca6a02a0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:51.032679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 185/387] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:51.033904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de00e29-258a-4594-a502-294f5d5e7947 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:51.037833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac6c86-278c-49bd-9221-2947db8f00a0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:51.038137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac6c86-278c-49bd-9221-2947db8f00a0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:51.038414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ac6c86-278c-49bd-9221-2947db8f00a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:51.047487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:51.064815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-91ac6c86-278c-49bd-9221-2947db8f00a0 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:51.066628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-91ac6c86-278c-49bd-9221-2947db8f00a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:51.095308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc8b1cf-f151-462e-8e54-b4bd6210cebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:51.095990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8b1cf-f151-462e-8e54-b4bd6210cebd None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:51.096267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8b1cf-f151-462e-8e54-b4bd6210cebd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:51.096539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc8b1cf-f151-462e-8e54-b4bd6210cebd None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014/groups/c780b10d545b439da0b83c42a939a617/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:51.098960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8b1cf-f151-462e-8e54-b4bd6210cebd None admin] RBAC: Authorizing `identity:create_grant(project_id=ab8379e77b244774a8bf8329866e9014, group_id=c780b10d545b439da0b83c42a939a617, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:51.100423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc8b1cf-f151-462e-8e54-b4bd6210cebd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:51.117281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 203/388] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Sep 13 06:47:50 2022] PUT /identity/v3/projects/ab8379e77b244774a8bf8329866e9014/groups/c780b10d545b439da0b83c42a939a617/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:51.118590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31a154-6c0a-4910-804d-f1ca79f5c8df None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:51.118873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31a154-6c0a-4910-804d-f1ca79f5c8df None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:51.119145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af31a154-6c0a-4910-804d-f1ca79f5c8df None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:51.122198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 204/389] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:51.128880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5098ed-e3fa-4454-8e44-107bfe15cf0a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:51.129204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5098ed-e3fa-4454-8e44-107bfe15cf0a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:51.129618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5098ed-e3fa-4454-8e44-107bfe15cf0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:51.149898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6b5098ed-e3fa-4454-8e44-107bfe15cf0a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:51.151189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6b5098ed-e3fa-4454-8e44-107bfe15cf0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:51.180363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 186/390] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:51.227087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 205/391] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:51.550671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac6c86-278c-49bd-9221-2947db8f00a0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:51.551230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5098ed-e3fa-4454-8e44-107bfe15cf0a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:51.580568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:51.690733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ac6c86-278c-49bd-9221-2947db8f00a0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:51.712192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:51.719149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5098ed-e3fa-4454-8e44-107bfe15cf0a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:51.760035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86db7d-867c-4e70-8e2b-5095a68afad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:51.760683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86db7d-867c-4e70-8e2b-5095a68afad0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:51.760964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86db7d-867c-4e70-8e2b-5095a68afad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:51.761225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86db7d-867c-4e70-8e2b-5095a68afad0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:51.782117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86db7d-867c-4e70-8e2b-5095a68afad0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6b86db7d-867c-4e70-8e2b-5095a68afad0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:51.786994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:51.787864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:51.788661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:51.792984 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 187/392] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:51.795844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86db7d-867c-4e70-8e2b-5095a68afad0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:51.796863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36015-4a0c-4226-89a0-7635d81e24bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:51.797510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36015-4a0c-4226-89a0-7635d81e24bf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:51.797782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36015-4a0c-4226-89a0-7635d81e24bf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:51.798047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da36015-4a0c-4226-89a0-7635d81e24bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:51.819431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da36015-4a0c-4226-89a0-7635d81e24bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7da36015-4a0c-4226-89a0-7635d81e24bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:51.824442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:51.825544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:51.827044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 206/393] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:51.848212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36015-4a0c-4226-89a0-7635d81e24bf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:51.975115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da36015-4a0c-4226-89a0-7635d81e24bf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:51.983749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:52.003955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86db7d-867c-4e70-8e2b-5095a68afad0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:52.010487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef12177-0595-413c-b2df-92dc1cf24547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.011100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef12177-0595-413c-b2df-92dc1cf24547 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.011377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef12177-0595-413c-b2df-92dc1cf24547 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.011643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef12177-0595-413c-b2df-92dc1cf24547 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.012308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef12177-0595-413c-b2df-92dc1cf24547 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:52.012676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef12177-0595-413c-b2df-92dc1cf24547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:52.014097 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef12177-0595-413c-b2df-92dc1cf24547 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:52.022925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 207/394] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:52.023919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ae137-4d07-42a7-9479-fc60c9b964e4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.024340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ae137-4d07-42a7-9479-fc60c9b964e4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.024522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30ae137-4d07-42a7-9479-fc60c9b964e4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.025745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 208/395] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:52.031154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74018b-a239-495e-aa6c-28d895a229b0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.031444 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74018b-a239-495e-aa6c-28d895a229b0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.032105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd74018b-a239-495e-aa6c-28d895a229b0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.032935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 209/396] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:52.033638 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74018b-a239-495e-aa6c-28d895a229b0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:52.047395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26700628-786e-4501-8c61-11cbf70275c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.048248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26700628-786e-4501-8c61-11cbf70275c6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.048707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26700628-786e-4501-8c61-11cbf70275c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.049282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26700628-786e-4501-8c61-11cbf70275c6 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.050248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26700628-786e-4501-8c61-11cbf70275c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:52.050694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26700628-786e-4501-8c61-11cbf70275c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:52.052346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26700628-786e-4501-8c61-11cbf70275c6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:52.053137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:52.065198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 188/397] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:52.066573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7b4ec-2da2-4bc0-8deb-65cd5920b06b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.066858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7b4ec-2da2-4bc0-8deb-65cd5920b06b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.067135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f7b4ec-2da2-4bc0-8deb-65cd5920b06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.087817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-59f7b4ec-2da2-4bc0-8deb-65cd5920b06b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:52.088983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-59f7b4ec-2da2-4bc0-8deb-65cd5920b06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:52.144425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd74018b-a239-495e-aa6c-28d895a229b0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:52.159485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 189/398] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:52.160534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a5120-4809-46ed-89f5-1a82622612dc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.160811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a5120-4809-46ed-89f5-1a82622612dc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.161076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542a5120-4809-46ed-89f5-1a82622612dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.178743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-542a5120-4809-46ed-89f5-1a82622612dc None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:52.179915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-542a5120-4809-46ed-89f5-1a82622612dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:52.202888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae66e3f-74ec-4560-ad0c-e7f6ce1257a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.204235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66e3f-74ec-4560-ad0c-e7f6ce1257a4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.204770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66e3f-74ec-4560-ad0c-e7f6ce1257a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.205513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae66e3f-74ec-4560-ad0c-e7f6ce1257a4 None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.207598 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae66e3f-74ec-4560-ad0c-e7f6ce1257a4 None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:52.210245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae66e3f-74ec-4560-ad0c-e7f6ce1257a4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:52.213193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 210/399] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:52 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:52.214291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae66e3f-74ec-4560-ad0c-e7f6ce1257a4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:52.271034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 190/400] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:52.272612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a5120-4809-46ed-89f5-1a82622612dc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:52.277607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:52.306344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:52.325964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae66e3f-74ec-4560-ad0c-e7f6ce1257a4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:52.361443 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977174b-bd23-4cc5-8aad-6ffb8abe52a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.362094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977174b-bd23-4cc5-8aad-6ffb8abe52a6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.362378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977174b-bd23-4cc5-8aad-6ffb8abe52a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.362926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4977174b-bd23-4cc5-8aad-6ffb8abe52a6 None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.363832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977174b-bd23-4cc5-8aad-6ffb8abe52a6 None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:52.365279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4977174b-bd23-4cc5-8aad-6ffb8abe52a6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:52.367380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 211/401] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:52 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:52.373580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977174b-bd23-4cc5-8aad-6ffb8abe52a6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:52.413778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542a5120-4809-46ed-89f5-1a82622612dc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:52.477506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43029ba2-9508-47f6-9c6f-dab2a3802d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.478166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43029ba2-9508-47f6-9c6f-dab2a3802d84 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.478446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43029ba2-9508-47f6-9c6f-dab2a3802d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.478715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43029ba2-9508-47f6-9c6f-dab2a3802d84 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.482580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43029ba2-9508-47f6-9c6f-dab2a3802d84 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:52.484286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43029ba2-9508-47f6-9c6f-dab2a3802d84 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:52.485943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 191/402] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:52 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:52.503702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43029ba2-9508-47f6-9c6f-dab2a3802d84 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:52.529515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4977174b-bd23-4cc5-8aad-6ffb8abe52a6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:52.570851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d64b71-ad86-46ff-a38f-f4a8f0dbbccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.571480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d64b71-ad86-46ff-a38f-f4a8f0dbbccc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.571756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d64b71-ad86-46ff-a38f-f4a8f0dbbccc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.572025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d64b71-ad86-46ff-a38f-f4a8f0dbbccc None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.573147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d64b71-ad86-46ff-a38f-f4a8f0dbbccc None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:52.574626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46d64b71-ad86-46ff-a38f-f4a8f0dbbccc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:52.576300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 212/403] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:52 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:52.577054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d64b71-ad86-46ff-a38f-f4a8f0dbbccc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:52.580409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:52.612920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:52.614551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43029ba2-9508-47f6-9c6f-dab2a3802d84 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:52.651187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda83352-87a4-496e-831a-527179beb47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.651842 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda83352-87a4-496e-831a-527179beb47e None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.652122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda83352-87a4-496e-831a-527179beb47e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.652664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda83352-87a4-496e-831a-527179beb47e None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819/groups/c780b10d545b439da0b83c42a939a617/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.655105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda83352-87a4-496e-831a-527179beb47e None admin] RBAC: Authorizing `identity:create_grant(project_id=fe3324999301468c8543ebc979dbd819, group_id=c780b10d545b439da0b83c42a939a617, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:52.656566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda83352-87a4-496e-831a-527179beb47e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:52.673075 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 192/404] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Sep 13 06:47:52 2022] PUT /identity/v3/projects/fe3324999301468c8543ebc979dbd819/groups/c780b10d545b439da0b83c42a939a617/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 06:47:52.675253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda83352-87a4-496e-831a-527179beb47e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:52.696016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d64b71-ad86-46ff-a38f-f4a8f0dbbccc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:52.735376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b5201-242a-4699-bb2a-e0fc496dc373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.736627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b5201-242a-4699-bb2a-e0fc496dc373 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.736627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b5201-242a-4699-bb2a-e0fc496dc373 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.736627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7b5201-242a-4699-bb2a-e0fc496dc373 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.749524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7b5201-242a-4699-bb2a-e0fc496dc373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2a7b5201-242a-4699-bb2a-e0fc496dc373 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:52.755370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:52.756365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 213/405] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:52.757749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b5201-242a-4699-bb2a-e0fc496dc373 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:52.815920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda83352-87a4-496e-831a-527179beb47e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:52.849440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:52.855288 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab316b0-710d-4e05-bada-0c2938ca2874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.856150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab316b0-710d-4e05-bada-0c2938ca2874 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.856400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab316b0-710d-4e05-bada-0c2938ca2874 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.857008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab316b0-710d-4e05-bada-0c2938ca2874 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014/groups/c780b10d545b439da0b83c42a939a617/roles/c622170611454f18bf01aa658a8cc06e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.860719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab316b0-710d-4e05-bada-0c2938ca2874 None admin] RBAC: Authorizing `identity:create_grant(project_id=ab8379e77b244774a8bf8329866e9014, group_id=c780b10d545b439da0b83c42a939a617, role_id=c622170611454f18bf01aa658a8cc06e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:52.863935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab316b0-710d-4e05-bada-0c2938ca2874 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:52.880422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 193/406] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Sep 13 06:47:52 2022] PUT /identity/v3/projects/ab8379e77b244774a8bf8329866e9014/groups/c780b10d545b439da0b83c42a939a617/roles/c622170611454f18bf01aa658a8cc06e => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:52.881094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab316b0-710d-4e05-bada-0c2938ca2874 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:52.891194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:52.920129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7b5201-242a-4699-bb2a-e0fc496dc373 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:52.964852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88027d-786a-4750-b731-5e7a321b8519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:52.966927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88027d-786a-4750-b731-5e7a321b8519 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:52.967547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88027d-786a-4750-b731-5e7a321b8519 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:52.968397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e88027d-786a-4750-b731-5e7a321b8519 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:52.984785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e88027d-786a-4750-b731-5e7a321b8519 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7e88027d-786a-4750-b731-5e7a321b8519 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:52.992311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:52.993406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:52.994374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:53.001425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 214/407] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:47:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:53.003565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88027d-786a-4750-b731-5e7a321b8519 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:53.047390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab316b0-710d-4e05-bada-0c2938ca2874 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:53.124891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea5f7-3296-4be1-bfaa-647ac2734072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:53.125919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea5f7-3296-4be1-bfaa-647ac2734072 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:53.126420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea5f7-3296-4be1-bfaa-647ac2734072 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:53.126885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eea5f7-3296-4be1-bfaa-647ac2734072 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:53.127748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eea5f7-3296-4be1-bfaa-647ac2734072 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:53.128268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eea5f7-3296-4be1-bfaa-647ac2734072 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:53.129846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eea5f7-3296-4be1-bfaa-647ac2734072 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:53.139824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 194/408] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:53.146378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea5f7-3296-4be1-bfaa-647ac2734072 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:53.165407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e88027d-786a-4750-b731-5e7a321b8519 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:53.205167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea5459-015a-4fa4-a4c5-5ec96b75c84f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:53.205821 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea5459-015a-4fa4-a4c5-5ec96b75c84f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:53.206163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea5459-015a-4fa4-a4c5-5ec96b75c84f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:53.206445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ea5459-015a-4fa4-a4c5-5ec96b75c84f None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:53.207119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea5459-015a-4fa4-a4c5-5ec96b75c84f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:53.207424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea5459-015a-4fa4-a4c5-5ec96b75c84f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:53.208827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ea5459-015a-4fa4-a4c5-5ec96b75c84f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:53.223092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 215/409] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:53.228104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea5459-015a-4fa4-a4c5-5ec96b75c84f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:53.233279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:53.252111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:53.287425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eea5f7-3296-4be1-bfaa-647ac2734072 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:53.329251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e523e-85c7-4cd4-bd32-ab86952cb03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:53.330410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e523e-85c7-4cd4-bd32-ab86952cb03e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:53.330885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e523e-85c7-4cd4-bd32-ab86952cb03e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:53.331737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283e523e-85c7-4cd4-bd32-ab86952cb03e None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:53.339084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e523e-85c7-4cd4-bd32-ab86952cb03e None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:53.340236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-283e523e-85c7-4cd4-bd32-ab86952cb03e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:53.342026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 195/410] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:53 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:53.347534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e523e-85c7-4cd4-bd32-ab86952cb03e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:53.454730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ea5459-015a-4fa4-a4c5-5ec96b75c84f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:53.478689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283e523e-85c7-4cd4-bd32-ab86952cb03e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:53.518476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:53.530698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2751e83-1427-45e9-9087-f873830d94db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:53.531647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751e83-1427-45e9-9087-f873830d94db None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:53.532126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751e83-1427-45e9-9087-f873830d94db None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:53.532594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2751e83-1427-45e9-9087-f873830d94db None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:53.534383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2751e83-1427-45e9-9087-f873830d94db None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:53.537855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2751e83-1427-45e9-9087-f873830d94db None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:53.538273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 196/411] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:53 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:53.543967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2751e83-1427-45e9-9087-f873830d94db None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:53.544886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd588d3-52c1-4dbd-bcdf-bc790d5bd1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:53.545553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd588d3-52c1-4dbd-bcdf-bc790d5bd1de None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:53.545837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd588d3-52c1-4dbd-bcdf-bc790d5bd1de None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:53.546132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd588d3-52c1-4dbd-bcdf-bc790d5bd1de None admin] PATH_INFO: `/v3/groups/504b2be6ba094c699f515c6e99cdbd40` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:53.547791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd588d3-52c1-4dbd-bcdf-bc790d5bd1de None admin] RBAC: Authorizing `identity:get_group(group_id=504b2be6ba094c699f515c6e99cdbd40)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:53.549411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd588d3-52c1-4dbd-bcdf-bc790d5bd1de None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:53.557340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 216/412] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:53 2022] GET /identity/v3/groups/504b2be6ba094c699f515c6e99cdbd40 => generated 229 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:53.560831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd588d3-52c1-4dbd-bcdf-bc790d5bd1de None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:53.673151 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:53.708565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd588d3-52c1-4dbd-bcdf-bc790d5bd1de None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:53.734651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2751e83-1427-45e9-9087-f873830d94db None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:53.746287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df20a3cb-885e-4a68-9d9c-2561bee6b800 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:53.747469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20a3cb-885e-4a68-9d9c-2561bee6b800 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:53.747944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20a3cb-885e-4a68-9d9c-2561bee6b800 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:53.748489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df20a3cb-885e-4a68-9d9c-2561bee6b800 None admin] PATH_INFO: `/v3/projects/2becfff2f1104b709c0a7babcac266dd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:53.750069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20a3cb-885e-4a68-9d9c-2561bee6b800 None admin] RBAC: Authorizing `identity:get_project(project_id=2becfff2f1104b709c0a7babcac266dd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:53.751865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df20a3cb-885e-4a68-9d9c-2561bee6b800 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:53.754125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 217/413] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:53 2022] GET /identity/v3/projects/2becfff2f1104b709c0a7babcac266dd => generated 344 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:53.755655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270f463-caed-446a-9fc8-5c86c1871811 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:53.756137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270f463-caed-446a-9fc8-5c86c1871811 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:53.756627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270f463-caed-446a-9fc8-5c86c1871811 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:53.758584 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 218/414] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:53.759613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270f463-caed-446a-9fc8-5c86c1871811 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:53.770922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f1dcb8-1d65-4a9a-b974-1492149784bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:53.771647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1dcb8-1d65-4a9a-b974-1492149784bf None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:53.771867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1dcb8-1d65-4a9a-b974-1492149784bf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:53.772143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f1dcb8-1d65-4a9a-b974-1492149784bf None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819/groups/c780b10d545b439da0b83c42a939a617/roles/c622170611454f18bf01aa658a8cc06e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:53.774384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1dcb8-1d65-4a9a-b974-1492149784bf None admin] RBAC: Authorizing `identity:create_grant(project_id=fe3324999301468c8543ebc979dbd819, group_id=c780b10d545b439da0b83c42a939a617, role_id=c622170611454f18bf01aa658a8cc06e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:53.775844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f1dcb8-1d65-4a9a-b974-1492149784bf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:53.793846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 197/415] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Sep 13 06:47:53 2022] PUT /identity/v3/projects/fe3324999301468c8543ebc979dbd819/groups/c780b10d545b439da0b83c42a939a617/roles/c622170611454f18bf01aa658a8cc06e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:53.798138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83410a-687c-476f-86dc-75cdda6c50d1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:53.798138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83410a-687c-476f-86dc-75cdda6c50d1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:53.798138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d83410a-687c-476f-86dc-75cdda6c50d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:53.819728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0d83410a-687c-476f-86dc-75cdda6c50d1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:53.821336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0d83410a-687c-476f-86dc-75cdda6c50d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:53.879762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:53.906552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 198/416] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:53.946940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270f463-caed-446a-9fc8-5c86c1871811 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:54.000363 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f4f38-aae0-4d34-ae77-0773fc665b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:54.001449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f4f38-aae0-4d34-ae77-0773fc665b09 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:54.001929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f4f38-aae0-4d34-ae77-0773fc665b09 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:54.002447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2f4f38-aae0-4d34-ae77-0773fc665b09 None admin] PATH_INFO: `/v3/projects/2becfff2f1104b709c0a7babcac266dd/groups/504b2be6ba094c699f515c6e99cdbd40/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:54.005864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f4f38-aae0-4d34-ae77-0773fc665b09 None admin] RBAC: Authorizing `identity:create_grant(project_id=2becfff2f1104b709c0a7babcac266dd, group_id=504b2be6ba094c699f515c6e99cdbd40, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:54.007691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2f4f38-aae0-4d34-ae77-0773fc665b09 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:54.030572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:54.036043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 219/417] 10.222.0.25 () {58 vars in 1442 bytes} [Tue Sep 13 06:47:53 2022] PUT /identity/v3/projects/2becfff2f1104b709c0a7babcac266dd/groups/504b2be6ba094c699f515c6e99cdbd40/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:47:54.180405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83410a-687c-476f-86dc-75cdda6c50d1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:54.296489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d83410a-687c-476f-86dc-75cdda6c50d1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:54.332041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823c777-75d5-4b9b-b9e4-f11bcbd8074f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:54.332745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c777-75d5-4b9b-b9e4-f11bcbd8074f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:54.333032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c777-75d5-4b9b-b9e4-f11bcbd8074f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:54.333409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0823c777-75d5-4b9b-b9e4-f11bcbd8074f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:54.344617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0823c777-75d5-4b9b-b9e4-f11bcbd8074f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0823c777-75d5-4b9b-b9e4-f11bcbd8074f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:54.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:54.350471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:54.351832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 199/418] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:54.356629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f4f38-aae0-4d34-ae77-0773fc665b09 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:54.495316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2f4f38-aae0-4d34-ae77-0773fc665b09 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:54.529810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498ace6-bd73-4aae-8968-1baa2580dd75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:54.530501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498ace6-bd73-4aae-8968-1baa2580dd75 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:54.530787 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498ace6-bd73-4aae-8968-1baa2580dd75 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:54.531061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3498ace6-bd73-4aae-8968-1baa2580dd75 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:54.531729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498ace6-bd73-4aae-8968-1baa2580dd75 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:54.532032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498ace6-bd73-4aae-8968-1baa2580dd75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:54.533415 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3498ace6-bd73-4aae-8968-1baa2580dd75 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:54.541691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 220/419] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:54.547761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823c777-75d5-4b9b-b9e4-f11bcbd8074f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:54.608106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:54.691081 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0823c777-75d5-4b9b-b9e4-f11bcbd8074f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:54.750808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a52ff-56db-46a0-b4a0-38f4ab784a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:54.751495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a52ff-56db-46a0-b4a0-38f4ab784a31 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:54.751739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a52ff-56db-46a0-b4a0-38f4ab784a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:54.751962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867a52ff-56db-46a0-b4a0-38f4ab784a31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:54.841794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a52ff-56db-46a0-b4a0-38f4ab784a31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:54.843299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867a52ff-56db-46a0-b4a0-38f4ab784a31 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:54.902452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:54.987990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 200/420] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:54.995241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498ace6-bd73-4aae-8968-1baa2580dd75 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:55.076946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:55.126377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3498ace6-bd73-4aae-8968-1baa2580dd75 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:55.174995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695790b-dcdc-4f08-91c4-56feb5c4146a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:55.176019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695790b-dcdc-4f08-91c4-56feb5c4146a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:55.176763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695790b-dcdc-4f08-91c4-56feb5c4146a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:55.177340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3695790b-dcdc-4f08-91c4-56feb5c4146a None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:55.181991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695790b-dcdc-4f08-91c4-56feb5c4146a None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:55.181991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3695790b-dcdc-4f08-91c4-56feb5c4146a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:55.182316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 221/421] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:54 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:55.188967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a52ff-56db-46a0-b4a0-38f4ab784a31 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:55.400503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867a52ff-56db-46a0-b4a0-38f4ab784a31 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:55.471557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fdf152-5288-47db-b861-402188ef9d3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:55.472214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdf152-5288-47db-b861-402188ef9d3e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:55.472495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdf152-5288-47db-b861-402188ef9d3e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:55.472771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fdf152-5288-47db-b861-402188ef9d3e None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:55.478745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdf152-5288-47db-b861-402188ef9d3e None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:55.480224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16fdf152-5288-47db-b861-402188ef9d3e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:55.482137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 201/422] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:55 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:55.489172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695790b-dcdc-4f08-91c4-56feb5c4146a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:55.537162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6963c-bb4d-4618-acc2-77592fe18e47 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:55.537498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6963c-bb4d-4618-acc2-77592fe18e47 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:55.537719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab6963c-bb4d-4618-acc2-77592fe18e47 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:55.539103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 202/423] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:47:55.545409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0240a47-d281-4824-a4c2-d97ac526e977 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:55.545698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0240a47-d281-4824-a4c2-d97ac526e977 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:55.545957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0240a47-d281-4824-a4c2-d97ac526e977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:55.587197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d0240a47-d281-4824-a4c2-d97ac526e977 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:55.588698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d0240a47-d281-4824-a4c2-d97ac526e977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:55.594385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:55.661291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3695790b-dcdc-4f08-91c4-56feb5c4146a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:55.706468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:55.709413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 203/424] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:55.715143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613f67-afd5-4a64-af93-e3fee06da3d0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:55.715434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613f67-afd5-4a64-af93-e3fee06da3d0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:55.715790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4613f67-afd5-4a64-af93-e3fee06da3d0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:55.717093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 204/425] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:55.719633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a687c-92cd-473a-a3d3-abe39b3aeb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:55.720254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a687c-92cd-473a-a3d3-abe39b3aeb82 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:55.720530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a687c-92cd-473a-a3d3-abe39b3aeb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:55.720803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8a687c-92cd-473a-a3d3-abe39b3aeb82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:55.721547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a687c-92cd-473a-a3d3-abe39b3aeb82 None admin] RBAC: Adding query filter params (group.id=c780b10d545b439da0b83c42a939a617, role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=ab8379e77b244774a8bf8329866e9014) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:55.721859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a687c-92cd-473a-a3d3-abe39b3aeb82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:55.723363 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8a687c-92cd-473a-a3d3-abe39b3aeb82 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:55.724877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf87f1-dbef-43b1-b38f-3ce7cf756c17 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:55.725162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf87f1-dbef-43b1-b38f-3ce7cf756c17 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:55.725436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cf87f1-dbef-43b1-b38f-3ce7cf756c17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:55.735960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 222/426] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:55 2022] GET /identity/v3/role_assignments?group.id=c780b10d545b439da0b83c42a939a617&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=2d59fcad66104464bbc848ef605f743e => generated 611 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:55.746396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e5cf87f1-dbef-43b1-b38f-3ce7cf756c17 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:55.747852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e5cf87f1-dbef-43b1-b38f-3ce7cf756c17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:55.836408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 205/427] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:56.076611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a687c-92cd-473a-a3d3-abe39b3aeb82 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:56.160211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf87f1-dbef-43b1-b38f-3ce7cf756c17 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:56.204825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8a687c-92cd-473a-a3d3-abe39b3aeb82 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:56.251689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ce225-979f-49fa-b4a6-685be571c146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:56.252569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ce225-979f-49fa-b4a6-685be571c146 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.252845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ce225-979f-49fa-b4a6-685be571c146 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.253112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99ce225-979f-49fa-b4a6-685be571c146 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:56.255525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:56.271277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99ce225-979f-49fa-b4a6-685be571c146 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a99ce225-979f-49fa-b4a6-685be571c146 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:56.278554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:56.280214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:56.281391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 223/428] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:56.288025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ce225-979f-49fa-b4a6-685be571c146 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:56.302182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cf87f1-dbef-43b1-b38f-3ce7cf756c17 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:56.346741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce56f7-e374-40a2-8a32-e33bbd3c5874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:56.347393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce56f7-e374-40a2-8a32-e33bbd3c5874 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.347670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce56f7-e374-40a2-8a32-e33bbd3c5874 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.347942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce56f7-e374-40a2-8a32-e33bbd3c5874 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:56.359968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:56.369544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce56f7-e374-40a2-8a32-e33bbd3c5874 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e3ce56f7-e374-40a2-8a32-e33bbd3c5874 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:56.375111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:56.376008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:56.376827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 206/429] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:47:56.377515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b0835-7c93-480f-9a94-d46e18777d19 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.377832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b0835-7c93-480f-9a94-d46e18777d19 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.379988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3b0835-7c93-480f-9a94-d46e18777d19 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:56.379988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 207/430] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:56.385775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20361e15-f2b1-459d-a1a6-f47a2873ef8c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.386063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20361e15-f2b1-459d-a1a6-f47a2873ef8c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.387001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20361e15-f2b1-459d-a1a6-f47a2873ef8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:56.410365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-20361e15-f2b1-459d-a1a6-f47a2873ef8c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:56.411561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-20361e15-f2b1-459d-a1a6-f47a2873ef8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:56.429281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99ce225-979f-49fa-b4a6-685be571c146 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:56.468502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b62e4-8a52-4665-a491-ae4e161f2e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:56.469118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b62e4-8a52-4665-a491-ae4e161f2e86 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.469562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b62e4-8a52-4665-a491-ae4e161f2e86 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.469982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148b62e4-8a52-4665-a491-ae4e161f2e86 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:56.471084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b62e4-8a52-4665-a491-ae4e161f2e86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:56.471400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b62e4-8a52-4665-a491-ae4e161f2e86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:56.472839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148b62e4-8a52-4665-a491-ae4e161f2e86 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:56.482698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 224/431] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:56.483562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b62e4-8a52-4665-a491-ae4e161f2e86 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:56.495588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 208/432] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:56.496748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20361e15-f2b1-459d-a1a6-f47a2873ef8c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:56.561183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:56.606866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20361e15-f2b1-459d-a1a6-f47a2873ef8c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:56.637586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148b62e4-8a52-4665-a491-ae4e161f2e86 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:56.643472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f350c-e7f9-425a-bb78-38c755e9ac5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:56.644361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f350c-e7f9-425a-bb78-38c755e9ac5a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.644802 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f350c-e7f9-425a-bb78-38c755e9ac5a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.645209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f350c-e7f9-425a-bb78-38c755e9ac5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:56.662480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:56.691531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f30acb-5d32-4cef-bbc2-9389110b1454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:56.692269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f30acb-5d32-4cef-bbc2-9389110b1454 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.692647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f30acb-5d32-4cef-bbc2-9389110b1454 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.692990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f30acb-5d32-4cef-bbc2-9389110b1454 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:56.693760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f30acb-5d32-4cef-bbc2-9389110b1454 None admin] RBAC: Adding query filter params (name=member) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:56.694161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f30acb-5d32-4cef-bbc2-9389110b1454 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:56.695666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f30acb-5d32-4cef-bbc2-9389110b1454 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:56.710119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 225/433] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:47:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:56.715046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f30acb-5d32-4cef-bbc2-9389110b1454 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:56.736447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f350c-e7f9-425a-bb78-38c755e9ac5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:56.737920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f350c-e7f9-425a-bb78-38c755e9ac5a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:56.802728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:56.817980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f30acb-5d32-4cef-bbc2-9389110b1454 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:56.846298 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 209/434] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:56.848203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf2238-82da-4da8-a56d-5f6cde2dda69 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.848485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf2238-82da-4da8-a56d-5f6cde2dda69 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.848755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedf2238-82da-4da8-a56d-5f6cde2dda69 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:56.850004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 210/435] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:47:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:47:56.850719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf2238-82da-4da8-a56d-5f6cde2dda69 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:56.857357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ebefb-d47d-499c-be7e-f36b4964875f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:56.857916 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ebefb-d47d-499c-be7e-f36b4964875f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.858230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ebefb-d47d-499c-be7e-f36b4964875f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.858509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ebefb-d47d-499c-be7e-f36b4964875f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:56.904742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:56.929196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ebefb-d47d-499c-be7e-f36b4964875f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:56.930731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ebefb-d47d-499c-be7e-f36b4964875f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:56.959136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedf2238-82da-4da8-a56d-5f6cde2dda69 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:56.992922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227ee81-2475-4602-9d81-8e2635cfbfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:56.993555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227ee81-2475-4602-9d81-8e2635cfbfa0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:56.993843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227ee81-2475-4602-9d81-8e2635cfbfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:56.994131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227ee81-2475-4602-9d81-8e2635cfbfa0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.004387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227ee81-2475-4602-9d81-8e2635cfbfa0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4227ee81-2475-4602-9d81-8e2635cfbfa0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:57.009297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:57.010186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:47:57.011019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 211/436] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:47:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:57.011680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227ee81-2475-4602-9d81-8e2635cfbfa0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.030767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:57.035477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 226/437] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:57.035477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d0c82-db9e-49b3-ae63-b9d7fc49eb4d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.035477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d0c82-db9e-49b3-ae63-b9d7fc49eb4d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.035477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4d0c82-db9e-49b3-ae63-b9d7fc49eb4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.051648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9a4d0c82-db9e-49b3-ae63-b9d7fc49eb4d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:47:57.052845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9a4d0c82-db9e-49b3-ae63-b9d7fc49eb4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:47:57.117888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227ee81-2475-4602-9d81-8e2635cfbfa0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.118266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 227/438] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:47:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:47:57.119085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d0c82-db9e-49b3-ae63-b9d7fc49eb4d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.128744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:57.154960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad8fdc-a88b-4f0b-9846-c8ce2ff76cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.155596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad8fdc-a88b-4f0b-9846-c8ce2ff76cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.155872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad8fdc-a88b-4f0b-9846-c8ce2ff76cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.156148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ad8fdc-a88b-4f0b-9846-c8ce2ff76cf1 None admin] PATH_INFO: `/v3/projects/ab8379e77b244774a8bf8329866e9014` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.157302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad8fdc-a88b-4f0b-9846-c8ce2ff76cf1 None admin] RBAC: Authorizing `identity:get_project(project_id=ab8379e77b244774a8bf8329866e9014)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:57.158809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ad8fdc-a88b-4f0b-9846-c8ce2ff76cf1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:57.160429 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 212/439] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/projects/ab8379e77b244774a8bf8329866e9014 => generated 298 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:57.161181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad8fdc-a88b-4f0b-9846-c8ce2ff76cf1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.227675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4d0c82-db9e-49b3-ae63-b9d7fc49eb4d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.252145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:57.262529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f716-8256-4b2b-a8df-c8e54062064b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.263352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f716-8256-4b2b-a8df-c8e54062064b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.263637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f716-8256-4b2b-a8df-c8e54062064b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.263890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c323f716-8256-4b2b-a8df-c8e54062064b None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.264704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323f716-8256-4b2b-a8df-c8e54062064b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:57.265011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323f716-8256-4b2b-a8df-c8e54062064b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:57.266407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c323f716-8256-4b2b-a8df-c8e54062064b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:57.268608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ad8fdc-a88b-4f0b-9846-c8ce2ff76cf1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.274489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 228/440] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:57.275427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f716-8256-4b2b-a8df-c8e54062064b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.307092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b1b0d-bf08-4604-8906-28e2e2a8b5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.307735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1b0d-bf08-4604-8906-28e2e2a8b5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.308366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1b0d-bf08-4604-8906-28e2e2a8b5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.308642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3b1b0d-bf08-4604-8906-28e2e2a8b5a6 None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.309965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b1b0d-bf08-4604-8906-28e2e2a8b5a6 None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:57.311498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3b1b0d-bf08-4604-8906-28e2e2a8b5a6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:57.313282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 213/441] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:57.313885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b1b0d-bf08-4604-8906-28e2e2a8b5a6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.352933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:57.386559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c323f716-8256-4b2b-a8df-c8e54062064b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.419836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e49cc-5a7a-467c-a152-b5728dc985b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.420465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e49cc-5a7a-467c-a152-b5728dc985b6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.420625 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3b1b0d-bf08-4604-8906-28e2e2a8b5a6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.420756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e49cc-5a7a-467c-a152-b5728dc985b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.420981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e49cc-5a7a-467c-a152-b5728dc985b6 None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.422102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e49cc-5a7a-467c-a152-b5728dc985b6 None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:57.423573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e49cc-5a7a-467c-a152-b5728dc985b6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:57.425126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 229/442] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:57.425879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e49cc-5a7a-467c-a152-b5728dc985b6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.456440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa64c84-6cd0-4d81-8755-9504c7508527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.457064 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa64c84-6cd0-4d81-8755-9504c7508527 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.457341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa64c84-6cd0-4d81-8755-9504c7508527 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.457614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa64c84-6cd0-4d81-8755-9504c7508527 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.480591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:57.524312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa64c84-6cd0-4d81-8755-9504c7508527 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:57.525781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa64c84-6cd0-4d81-8755-9504c7508527 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:57.527386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e49cc-5a7a-467c-a152-b5728dc985b6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.562390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429033a-495d-49dc-ba6a-d93b669f7c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.563028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429033a-495d-49dc-ba6a-d93b669f7c8d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.563309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429033a-495d-49dc-ba6a-d93b669f7c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.563586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9429033a-495d-49dc-ba6a-d93b669f7c8d None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.564701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429033a-495d-49dc-ba6a-d93b669f7c8d None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:57.566186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9429033a-495d-49dc-ba6a-d93b669f7c8d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:57.567775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 230/443] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:57.568541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429033a-495d-49dc-ba6a-d93b669f7c8d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.577501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:57.624085 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 214/444] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:47:57.624917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa64c84-6cd0-4d81-8755-9504c7508527 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.671661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9429033a-495d-49dc-ba6a-d93b669f7c8d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.679010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:57.705156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86acbe1-7593-455e-bb75-caeb729d93f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.705789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86acbe1-7593-455e-bb75-caeb729d93f7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.706062 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86acbe1-7593-455e-bb75-caeb729d93f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.706358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86acbe1-7593-455e-bb75-caeb729d93f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.716098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86acbe1-7593-455e-bb75-caeb729d93f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d86acbe1-7593-455e-bb75-caeb729d93f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:47:57.721246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:47:57.722283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:47:57.723076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 231/445] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:47:57.723730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86acbe1-7593-455e-bb75-caeb729d93f7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.728313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa64c84-6cd0-4d81-8755-9504c7508527 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.762680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cad0d-f183-41d0-928a-a7e732dadc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.763311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cad0d-f183-41d0-928a-a7e732dadc83 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.763590 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cad0d-f183-41d0-928a-a7e732dadc83 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.763859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9cad0d-f183-41d0-928a-a7e732dadc83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.764579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cad0d-f183-41d0-928a-a7e732dadc83 None admin] RBAC: Adding query filter params (group.id=c780b10d545b439da0b83c42a939a617, role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=ab8379e77b244774a8bf8329866e9014) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:57.764886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cad0d-f183-41d0-928a-a7e732dadc83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:57.766360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9cad0d-f183-41d0-928a-a7e732dadc83 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:57.771954 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 215/446] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/role_assignments?group.id=c780b10d545b439da0b83c42a939a617&scope.project.id=ab8379e77b244774a8bf8329866e9014&role.id=c622170611454f18bf01aa658a8cc06e => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:57.772731 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cad0d-f183-41d0-928a-a7e732dadc83 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.787357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:57.828215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86acbe1-7593-455e-bb75-caeb729d93f7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.866270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ae669-784e-4c92-9ffc-e2ab332a8bbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.867227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae669-784e-4c92-9ffc-e2ab332a8bbd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.867507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae669-784e-4c92-9ffc-e2ab332a8bbd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.867780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42ae669-784e-4c92-9ffc-e2ab332a8bbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.868533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ae669-784e-4c92-9ffc-e2ab332a8bbd None admin] RBAC: Adding query filter params (group.id=c780b10d545b439da0b83c42a939a617, role.id=2d59fcad66104464bbc848ef605f743e, scope.project.id=fe3324999301468c8543ebc979dbd819) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:57.868844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ae669-784e-4c92-9ffc-e2ab332a8bbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:57.870437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42ae669-784e-4c92-9ffc-e2ab332a8bbd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:57.882111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 232/447] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/role_assignments?group.id=c780b10d545b439da0b83c42a939a617&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=2d59fcad66104464bbc848ef605f743e => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:57.882111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ae669-784e-4c92-9ffc-e2ab332a8bbd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:57.920079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9cad0d-f183-41d0-928a-a7e732dadc83 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:57.942606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:57.967291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aa581-ea05-4181-a7cd-ee032440e05f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:57.967977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa581-ea05-4181-a7cd-ee032440e05f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:57.968261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa581-ea05-4181-a7cd-ee032440e05f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:57.968539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4aa581-ea05-4181-a7cd-ee032440e05f None admin] PATH_INFO: `/v3/projects/fe3324999301468c8543ebc979dbd819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:57.969779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aa581-ea05-4181-a7cd-ee032440e05f None admin] RBAC: Authorizing `identity:get_project(project_id=fe3324999301468c8543ebc979dbd819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:57.971315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4aa581-ea05-4181-a7cd-ee032440e05f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:57.973833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 216/448] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/projects/fe3324999301468c8543ebc979dbd819 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:57.980345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aa581-ea05-4181-a7cd-ee032440e05f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:58.005409 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42ae669-784e-4c92-9ffc-e2ab332a8bbd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:58.045639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:58.052187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a0114-06b5-4d14-8a4f-4ce81b77772a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:58.052823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a0114-06b5-4d14-8a4f-4ce81b77772a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:58.053280 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a0114-06b5-4d14-8a4f-4ce81b77772a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:58.053561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325a0114-06b5-4d14-8a4f-4ce81b77772a None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:58.054705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a0114-06b5-4d14-8a4f-4ce81b77772a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:58.055016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a0114-06b5-4d14-8a4f-4ce81b77772a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:58.056439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325a0114-06b5-4d14-8a4f-4ce81b77772a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:58.067989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 233/449] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:58.074605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a0114-06b5-4d14-8a4f-4ce81b77772a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:58.120777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4aa581-ea05-4181-a7cd-ee032440e05f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:58.167575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128965-8fe3-4a0d-8715-6e5807b704a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:58.168217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128965-8fe3-4a0d-8715-6e5807b704a9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:58.168492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128965-8fe3-4a0d-8715-6e5807b704a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:58.168794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94128965-8fe3-4a0d-8715-6e5807b704a9 None admin] PATH_INFO: `/v3/groups/c780b10d545b439da0b83c42a939a617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:58.170385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94128965-8fe3-4a0d-8715-6e5807b704a9 None admin] RBAC: Authorizing `identity:get_group(group_id=c780b10d545b439da0b83c42a939a617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:58.172209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94128965-8fe3-4a0d-8715-6e5807b704a9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:58.174368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 217/450] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:57 2022] GET /identity/v3/groups/c780b10d545b439da0b83c42a939a617 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:58.180157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128965-8fe3-4a0d-8715-6e5807b704a9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:58.196888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325a0114-06b5-4d14-8a4f-4ce81b77772a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:58.237268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89354ebc-5c35-4084-a89c-402db39b445a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:58.238247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89354ebc-5c35-4084-a89c-402db39b445a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:58.238527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89354ebc-5c35-4084-a89c-402db39b445a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:58.238813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89354ebc-5c35-4084-a89c-402db39b445a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:58.246863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:58.303512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:58.305007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94128965-8fe3-4a0d-8715-6e5807b704a9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:58.316206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89354ebc-5c35-4084-a89c-402db39b445a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:58.317748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89354ebc-5c35-4084-a89c-402db39b445a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:58.345784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea93714-4ff2-4a92-b485-9a3e63c5c602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:58.346456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea93714-4ff2-4a92-b485-9a3e63c5c602 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:58.346721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea93714-4ff2-4a92-b485-9a3e63c5c602 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:58.346998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea93714-4ff2-4a92-b485-9a3e63c5c602 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:58.347729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea93714-4ff2-4a92-b485-9a3e63c5c602 None admin] RBAC: Adding query filter params (group.id=c780b10d545b439da0b83c42a939a617, role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=fe3324999301468c8543ebc979dbd819) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:58.348048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea93714-4ff2-4a92-b485-9a3e63c5c602 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:58.349477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea93714-4ff2-4a92-b485-9a3e63c5c602 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:58.356664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 218/451] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:58 2022] GET /identity/v3/role_assignments?group.id=c780b10d545b439da0b83c42a939a617&scope.project.id=fe3324999301468c8543ebc979dbd819&role.id=c622170611454f18bf01aa658a8cc06e => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:47:58.429372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 234/452] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:47:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 06:47:58.435678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea93714-4ff2-4a92-b485-9a3e63c5c602 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:58.543128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea93714-4ff2-4a92-b485-9a3e63c5c602 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:58.562335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:58.580847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d37cb4-c365-4bb1-bb27-4a097f0314a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:58.581482 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d37cb4-c365-4bb1-bb27-4a097f0314a8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:58.581757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d37cb4-c365-4bb1-bb27-4a097f0314a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:58.582025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d37cb4-c365-4bb1-bb27-4a097f0314a8 None admin] PATH_INFO: `/v3/projects/2becfff2f1104b709c0a7babcac266dd` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:58.583191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d37cb4-c365-4bb1-bb27-4a097f0314a8 None admin] RBAC: Authorizing `identity:get_project(project_id=2becfff2f1104b709c0a7babcac266dd)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:58.584657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d37cb4-c365-4bb1-bb27-4a097f0314a8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:58.586300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 219/453] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:47:58 2022] GET /identity/v3/projects/2becfff2f1104b709c0a7babcac266dd => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:58.592079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89354ebc-5c35-4084-a89c-402db39b445a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:58.687746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:47:58.696601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89354ebc-5c35-4084-a89c-402db39b445a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:58.729811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d888f0-10d9-4181-90d2-dcaade6ed9b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:58.730486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d888f0-10d9-4181-90d2-dcaade6ed9b8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:58.730754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d888f0-10d9-4181-90d2-dcaade6ed9b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:58.731021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d888f0-10d9-4181-90d2-dcaade6ed9b8 None admin] PATH_INFO: `/v3/groups/504b2be6ba094c699f515c6e99cdbd40` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:58.732195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d888f0-10d9-4181-90d2-dcaade6ed9b8 None admin] RBAC: Authorizing `identity:get_group(group_id=504b2be6ba094c699f515c6e99cdbd40)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:58.733650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d888f0-10d9-4181-90d2-dcaade6ed9b8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:58.735273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 235/454] 10.222.0.25 () {56 vars in 1180 bytes} [Tue Sep 13 06:47:58 2022] GET /identity/v3/groups/504b2be6ba094c699f515c6e99cdbd40 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:47:58.741437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d37cb4-c365-4bb1-bb27-4a097f0314a8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:47:58.842856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d37cb4-c365-4bb1-bb27-4a097f0314a8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:47:58.877957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1458d4e2-f27f-4fd2-ad05-2db265ea10bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:47:58.878580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d4e2-f27f-4fd2-ad05-2db265ea10bd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:47:58.878857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d4e2-f27f-4fd2-ad05-2db265ea10bd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:47:58.879125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1458d4e2-f27f-4fd2-ad05-2db265ea10bd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:47:58.879850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458d4e2-f27f-4fd2-ad05-2db265ea10bd None admin] RBAC: Adding query filter params (group.id=504b2be6ba094c699f515c6e99cdbd40, role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=2becfff2f1104b709c0a7babcac266dd) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:47:58.880153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458d4e2-f27f-4fd2-ad05-2db265ea10bd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:47:58.881580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1458d4e2-f27f-4fd2-ad05-2db265ea10bd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:47:58.887978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 220/455] 10.222.0.25 () {56 vars in 1376 bytes} [Tue Sep 13 06:47:58 2022] GET /identity/v3/role_assignments?group.id=504b2be6ba094c699f515c6e99cdbd40&scope.project.id=2becfff2f1104b709c0a7babcac266dd&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 611 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:01.707657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2873391-d62c-406c-916f-3c8af44d3e88 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:01.708543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2873391-d62c-406c-916f-3c8af44d3e88 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:01.708959 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2873391-d62c-406c-916f-3c8af44d3e88 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:01.710825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 236/456] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:01.716044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73980be1-803e-4e00-b94c-44ad8a92decc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:01.716834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73980be1-803e-4e00-b94c-44ad8a92decc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:01.717251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73980be1-803e-4e00-b94c-44ad8a92decc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:01.747329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-73980be1-803e-4e00-b94c-44ad8a92decc None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:01.748821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-73980be1-803e-4e00-b94c-44ad8a92decc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:01.751795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74004c7-1ae8-4c9f-87dd-1fb9435f15cf None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:01.752327 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74004c7-1ae8-4c9f-87dd-1fb9435f15cf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:01.752804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74004c7-1ae8-4c9f-87dd-1fb9435f15cf None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:01.754570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 237/457] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:01.760064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc28a66-00e5-494c-957c-7e12f4730a98 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:01.760578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc28a66-00e5-494c-957c-7e12f4730a98 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:01.761039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc28a66-00e5-494c-957c-7e12f4730a98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:01.786683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bcc28a66-00e5-494c-957c-7e12f4730a98 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:01.787878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bcc28a66-00e5-494c-957c-7e12f4730a98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:01.825870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:01.864103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 221/458] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:01.908703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 238/459] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:02.179401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73980be1-803e-4e00-b94c-44ad8a92decc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:02.187784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1d272-4c5d-45b5-9c1c-3eefa0bea3a1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:02.187784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1d272-4c5d-45b5-9c1c-3eefa0bea3a1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:02.187784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d1d272-4c5d-45b5-9c1c-3eefa0bea3a1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:02.188437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 239/460] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:02.193081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb79a5d-fe3e-44aa-be60-d2a462fae131 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:02.193415 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb79a5d-fe3e-44aa-be60-d2a462fae131 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:02.193639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb79a5d-fe3e-44aa-be60-d2a462fae131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:02.220054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dbb79a5d-fe3e-44aa-be60-d2a462fae131 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:02.221266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dbb79a5d-fe3e-44aa-be60-d2a462fae131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:02.313679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 240/461] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:48:02.314579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb79a5d-fe3e-44aa-be60-d2a462fae131 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:02.329731 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73980be1-803e-4e00-b94c-44ad8a92decc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:02.366519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:02.403366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f9822-ad7f-4854-b0dc-7bcdf9b6a032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:02.404094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f9822-ad7f-4854-b0dc-7bcdf9b6a032 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:02.404374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f9822-ad7f-4854-b0dc-7bcdf9b6a032 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:02.404646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba6f9822-ad7f-4854-b0dc-7bcdf9b6a032 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:02.423156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f9822-ad7f-4854-b0dc-7bcdf9b6a032 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:02.424820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba6f9822-ad7f-4854-b0dc-7bcdf9b6a032 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ba6f9822-ad7f-4854-b0dc-7bcdf9b6a032 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:02.451618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:02.452569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:02.453420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:02.453420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:02.453420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:02.453420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 222/462] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:02.462285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f9822-ad7f-4854-b0dc-7bcdf9b6a032 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:02.527875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:02.530862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb79a5d-fe3e-44aa-be60-d2a462fae131 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:02.596463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320883a3-e24b-4ca3-be76-b7c901028f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:02.597119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320883a3-e24b-4ca3-be76-b7c901028f9e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:02.597572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320883a3-e24b-4ca3-be76-b7c901028f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:02.597847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320883a3-e24b-4ca3-be76-b7c901028f9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:02.612475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320883a3-e24b-4ca3-be76-b7c901028f9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:02.614009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320883a3-e24b-4ca3-be76-b7c901028f9e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:02.631956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba6f9822-ad7f-4854-b0dc-7bcdf9b6a032 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-320883a3-e24b-4ca3-be76-b7c901028f9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:02.635182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:02.636100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:02.637452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:02.637452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:02.637452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:02.637452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 241/463] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:02.637789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a2bd7-b52a-4751-aa2c-de08ee1c476e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:02.638135 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a2bd7-b52a-4751-aa2c-de08ee1c476e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:02.642500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a2bd7-b52a-4751-aa2c-de08ee1c476e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:02.644220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 242/464] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:02 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:02.645169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a2bd7-b52a-4751-aa2c-de08ee1c476e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:02.669811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4c910-ac75-4c04-a4d4-396ae2dddae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:02.670792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4c910-ac75-4c04-a4d4-396ae2dddae8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:02.671268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4c910-ac75-4c04-a4d4-396ae2dddae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:02.671788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd4c910-ac75-4c04-a4d4-396ae2dddae8 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:02.672529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4c910-ac75-4c04-a4d4-396ae2dddae8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:02.672929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4c910-ac75-4c04-a4d4-396ae2dddae8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:02.674426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd4c910-ac75-4c04-a4d4-396ae2dddae8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:02.691955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 223/465] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:02.691955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4c910-ac75-4c04-a4d4-396ae2dddae8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:02.725911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:02.759228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a2bd7-b52a-4751-aa2c-de08ee1c476e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:02.790329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:02.797367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6417aad-235c-4968-88e2-f3990e616a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:02.798007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6417aad-235c-4968-88e2-f3990e616a61 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:02.798324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6417aad-235c-4968-88e2-f3990e616a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:02.798602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6417aad-235c-4968-88e2-f3990e616a61 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:02.800071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd4c910-ac75-4c04-a4d4-396ae2dddae8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:02.810593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6417aad-235c-4968-88e2-f3990e616a61 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:02.812083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6417aad-235c-4968-88e2-f3990e616a61 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c6417aad-235c-4968-88e2-f3990e616a61 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:02.827583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:02.828597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:02.829456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:02.829456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:02.829456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:02.829456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 243/466] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:02.830162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1cd4e-5fba-42c7-9124-bc0cef9f48d4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:02.830325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1cd4e-5fba-42c7-9124-bc0cef9f48d4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:02.830565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf1cd4e-5fba-42c7-9124-bc0cef9f48d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:02.849153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1d334-c68a-42b3-ac2b-2e5db42fd086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:02.849613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1cf1cd4e-5fba-42c7-9124-bc0cef9f48d4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:02.850752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1d334-c68a-42b3-ac2b-2e5db42fd086 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:02.850859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1cf1cd4e-5fba-42c7-9124-bc0cef9f48d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:02.851342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1d334-c68a-42b3-ac2b-2e5db42fd086 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:02.851817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f1d334-c68a-42b3-ac2b-2e5db42fd086 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:02.852679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1d334-c68a-42b3-ac2b-2e5db42fd086 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:02.853129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1d334-c68a-42b3-ac2b-2e5db42fd086 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:02.854598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f1d334-c68a-42b3-ac2b-2e5db42fd086 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:02.873467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 224/467] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:02.874935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1d334-c68a-42b3-ac2b-2e5db42fd086 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:02.934707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 244/468] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:02.936162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1cd4e-5fba-42c7-9124-bc0cef9f48d4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:02.986794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f1d334-c68a-42b3-ac2b-2e5db42fd086 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:02.995920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:03.022545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cf9ae-b30e-4357-b39d-d494704dd52b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:03.023155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cf9ae-b30e-4357-b39d-d494704dd52b None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.023440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cf9ae-b30e-4357-b39d-d494704dd52b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.023711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2cf9ae-b30e-4357-b39d-d494704dd52b None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.024368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2cf9ae-b30e-4357-b39d-d494704dd52b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:03.025836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2cf9ae-b30e-4357-b39d-d494704dd52b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:03.045169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf1cd4e-5fba-42c7-9124-bc0cef9f48d4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:03.062110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 225/469] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:48:02 2022] POST /identity/v3/users => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:03.062110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cf9ae-b30e-4357-b39d-d494704dd52b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:03.076158 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:03.094980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ce2854-8785-41c5-af1f-ca04db5efffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:03.095729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce2854-8785-41c5-af1f-ca04db5efffb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.096073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce2854-8785-41c5-af1f-ca04db5efffb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.096421 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ce2854-8785-41c5-af1f-ca04db5efffb None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.097146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce2854-8785-41c5-af1f-ca04db5efffb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:03.097515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce2854-8785-41c5-af1f-ca04db5efffb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:03.099213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ce2854-8785-41c5-af1f-ca04db5efffb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:03.112145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 245/470] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:03.113641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822faf7a-4795-4cae-ba0b-b0d35d0edae1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.113997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822faf7a-4795-4cae-ba0b-b0d35d0edae1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.114370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822faf7a-4795-4cae-ba0b-b0d35d0edae1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.120149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 246/471] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:03.121167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822faf7a-4795-4cae-ba0b-b0d35d0edae1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:03.195715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2cf9ae-b30e-4357-b39d-d494704dd52b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:03.237065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c28c69-bf4a-4bb5-94c9-a1ad2f088ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:03.238147 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c28c69-bf4a-4bb5-94c9-a1ad2f088ac0 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.238442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c28c69-bf4a-4bb5-94c9-a1ad2f088ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.238722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c28c69-bf4a-4bb5-94c9-a1ad2f088ac0 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.239737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c28c69-bf4a-4bb5-94c9-a1ad2f088ac0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:03.241898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c28c69-bf4a-4bb5-94c9-a1ad2f088ac0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:03.251188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822faf7a-4795-4cae-ba0b-b0d35d0edae1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:03.267159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:03.271125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 226/472] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:48:03 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:03.272543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa70a3c-a0b1-449c-a8e8-22eb63fa4e63 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.272997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa70a3c-a0b1-449c-a8e8-22eb63fa4e63 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.273415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa70a3c-a0b1-449c-a8e8-22eb63fa4e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.292747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6bafb-f85a-45f8-958b-fe4c9a8f25d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:03.293380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6bafb-f85a-45f8-958b-fe4c9a8f25d1 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.293730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6bafb-f85a-45f8-958b-fe4c9a8f25d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.294004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c6bafb-f85a-45f8-958b-fe4c9a8f25d1 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.294703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6bafb-f85a-45f8-958b-fe4c9a8f25d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:03.294985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4aa70a3c-a0b1-449c-a8e8-22eb63fa4e63 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:03.296207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c6bafb-f85a-45f8-958b-fe4c9a8f25d1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:03.296316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4aa70a3c-a0b1-449c-a8e8-22eb63fa4e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:03.331282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 247/473] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:48:03 2022] POST /identity/v3/users => generated 248 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:03.338275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6bafb-f85a-45f8-958b-fe4c9a8f25d1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:03.390969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:03.413322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 227/474] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:03.494534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c6bafb-f85a-45f8-958b-fe4c9a8f25d1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:03.546466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90c2fd-952f-44e3-97d7-8366c83ab9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:03.547120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90c2fd-952f-44e3-97d7-8366c83ab9c2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.547395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90c2fd-952f-44e3-97d7-8366c83ab9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.547661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f90c2fd-952f-44e3-97d7-8366c83ab9c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.560869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90c2fd-952f-44e3-97d7-8366c83ab9c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:03.562375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f90c2fd-952f-44e3-97d7-8366c83ab9c2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5f90c2fd-952f-44e3-97d7-8366c83ab9c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:03.577751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:03.578741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:03.579627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:03.579627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:03.579627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:03.579627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 248/475] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:03.585394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa70a3c-a0b1-449c-a8e8-22eb63fa4e63 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:03.703817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90c2fd-952f-44e3-97d7-8366c83ab9c2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:03.704039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa70a3c-a0b1-449c-a8e8-22eb63fa4e63 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:03.734851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:03.750265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3efec1-546a-4b97-a61b-76282ccd338c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:03.750821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3efec1-546a-4b97-a61b-76282ccd338c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.751103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3efec1-546a-4b97-a61b-76282ccd338c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.751375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3efec1-546a-4b97-a61b-76282ccd338c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.752588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3efec1-546a-4b97-a61b-76282ccd338c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:03.752588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3efec1-546a-4b97-a61b-76282ccd338c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:03.753928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3efec1-546a-4b97-a61b-76282ccd338c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:03.772568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 228/476] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:03.772568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3efec1-546a-4b97-a61b-76282ccd338c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:03.852106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:03.898777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f90c2fd-952f-44e3-97d7-8366c83ab9c2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:03.917459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3efec1-546a-4b97-a61b-76282ccd338c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:03.967999 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:03.968622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.968897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.969167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.969854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:03.971561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:03.986105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e48825-5eb8-4813-92f1-2522e3f0f35b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:03.987360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e48825-5eb8-4813-92f1-2522e3f0f35b None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:03.987360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e48825-5eb8-4813-92f1-2522e3f0f35b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:03.987526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e48825-5eb8-4813-92f1-2522e3f0f35b None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:03.988184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e48825-5eb8-4813-92f1-2522e3f0f35b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:03.989689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e48825-5eb8-4813-92f1-2522e3f0f35b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:03.993836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.sql.core [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 06:48:03.993836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 06:48:03.993836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [parameters: {'id': '85a72e90327d4790a0629b74266b3396', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 06:48:03.993836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 13 06:48:04.013588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return ref.to_dict() Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 06:48:04.014552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application yield resource Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise self.value Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application session.commit() Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 06:48:04.015489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.flush() Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application compat.raise_( Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 06:48:04.016369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 06:48:04.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 06:48:04.018110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [parameters: {'id': '85a72e90327d4790a0629b74266b3396', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:04.018918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 13 06:48:04.019723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:04.022572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 249/477] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Sep 13 06:48:03 2022] POST /identity/v3/roles => generated 144 bytes in 312 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:04.022572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:04.044407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 229/478] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:48:03 2022] POST /identity/v3/users => generated 249 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:48:04.161030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:04.180270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd72ed-970c-498b-a251-aad746d0e1d8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:04.232542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1eabe7-bbbf-43f2-9145-cf163f5b76aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:04.233213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1eabe7-bbbf-43f2-9145-cf163f5b76aa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:04.233496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1eabe7-bbbf-43f2-9145-cf163f5b76aa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:04.233775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1eabe7-bbbf-43f2-9145-cf163f5b76aa None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:04.252111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1eabe7-bbbf-43f2-9145-cf163f5b76aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-eb1eabe7-bbbf-43f2-9145-cf163f5b76aa None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:04.257544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:04.259585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:04.261505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 250/479] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:04.270125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e48825-5eb8-4813-92f1-2522e3f0f35b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:04.380433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e48825-5eb8-4813-92f1-2522e3f0f35b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:04.391930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:04.419130 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd53dd7e-a587-4f8c-8e8e-b673e57305ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:04.420004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53dd7e-a587-4f8c-8e8e-b673e57305ea None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:04.420426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53dd7e-a587-4f8c-8e8e-b673e57305ea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:04.420841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd53dd7e-a587-4f8c-8e8e-b673e57305ea None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:04.421683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53dd7e-a587-4f8c-8e8e-b673e57305ea None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:04.422149 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53dd7e-a587-4f8c-8e8e-b673e57305ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:04.423675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd53dd7e-a587-4f8c-8e8e-b673e57305ea None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:04.438332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 230/480] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:05.748575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d2597-32b1-44a5-9023-939826bf7c8b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:05.749641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d2597-32b1-44a5-9023-939826bf7c8b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:05.750192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116d2597-32b1-44a5-9023-939826bf7c8b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:05.752045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 251/481] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:05.757073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f573ec7-4417-4516-983d-75cd9b2fa1c3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:05.757963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f573ec7-4417-4516-983d-75cd9b2fa1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:05.758471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f573ec7-4417-4516-983d-75cd9b2fa1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:05.784546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5f573ec7-4417-4516-983d-75cd9b2fa1c3 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:05.786131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5f573ec7-4417-4516-983d-75cd9b2fa1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:05.868629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 231/482] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:06.073221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c89a3-5c83-4739-bf63-f9d1a97aa1a8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:06.073734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c89a3-5c83-4739-bf63-f9d1a97aa1a8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:06.074185 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c89a3-5c83-4739-bf63-f9d1a97aa1a8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:06.076273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 252/483] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:06.081998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af17d61-3f3c-467b-ba35-40b8ca4ebd40 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:06.082317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af17d61-3f3c-467b-ba35-40b8ca4ebd40 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:06.082592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af17d61-3f3c-467b-ba35-40b8ca4ebd40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:06.111086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3af17d61-3f3c-467b-ba35-40b8ca4ebd40 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:06.112290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3af17d61-3f3c-467b-ba35-40b8ca4ebd40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:06.155328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b37eb1-ac82-4a51-8474-712ec058904d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:06.155525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b37eb1-ac82-4a51-8474-712ec058904d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:06.155795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b37eb1-ac82-4a51-8474-712ec058904d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:06.157366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 253/484] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:06.162437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a746fd1-a265-4dae-b5f7-4369d6908d7d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:06.162721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a746fd1-a265-4dae-b5f7-4369d6908d7d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:06.162996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a746fd1-a265-4dae-b5f7-4369d6908d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:06.183865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2a746fd1-a265-4dae-b5f7-4369d6908d7d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:06.185175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2a746fd1-a265-4dae-b5f7-4369d6908d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:06.211923 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 232/485] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:06.255439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af17d61-3f3c-467b-ba35-40b8ca4ebd40 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:06.280551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 254/486] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:48:06.315094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:06.419161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af17d61-3f3c-467b-ba35-40b8ca4ebd40 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:06.465139 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64782422-5cb0-4544-8533-9934a3d11ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:06.466030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64782422-5cb0-4544-8533-9934a3d11ea9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:06.466331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64782422-5cb0-4544-8533-9934a3d11ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:06.466599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64782422-5cb0-4544-8533-9934a3d11ea9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:06.480737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64782422-5cb0-4544-8533-9934a3d11ea9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-64782422-5cb0-4544-8533-9934a3d11ea9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:06.486066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:06.486992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:06.488249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 233/487] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:06.495101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a746fd1-a265-4dae-b5f7-4369d6908d7d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:06.614426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:06.626118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64782422-5cb0-4544-8533-9934a3d11ea9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:06.633513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a746fd1-a265-4dae-b5f7-4369d6908d7d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:06.673094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920297f-2a8c-4be2-a9a1-e72852cdb9c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:06.673775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920297f-2a8c-4be2-a9a1-e72852cdb9c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:06.674051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920297f-2a8c-4be2-a9a1-e72852cdb9c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:06.674362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920297f-2a8c-4be2-a9a1-e72852cdb9c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:06.675125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920297f-2a8c-4be2-a9a1-e72852cdb9c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:06.675431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920297f-2a8c-4be2-a9a1-e72852cdb9c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:06.676930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920297f-2a8c-4be2-a9a1-e72852cdb9c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:06.686786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 255/488] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:06.687721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920297f-2a8c-4be2-a9a1-e72852cdb9c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:06.729629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:06.739214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64782422-5cb0-4544-8533-9934a3d11ea9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:06.782458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d474af0-8d1a-4d28-936c-ca6bf8f5256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:06.783152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d474af0-8d1a-4d28-936c-ca6bf8f5256f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:06.783377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d474af0-8d1a-4d28-936c-ca6bf8f5256f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:06.783612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d474af0-8d1a-4d28-936c-ca6bf8f5256f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:06.799924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d474af0-8d1a-4d28-936c-ca6bf8f5256f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:06.800632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920297f-2a8c-4be2-a9a1-e72852cdb9c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1d474af0-8d1a-4d28-936c-ca6bf8f5256f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:06.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:06.807055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:06.808223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:06.810991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 234/489] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:06.812121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d474af0-8d1a-4d28-936c-ca6bf8f5256f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:06.839671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606a5d-30af-43dd-8e5b-461ec04048f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:06.840355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606a5d-30af-43dd-8e5b-461ec04048f1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:06.840630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606a5d-30af-43dd-8e5b-461ec04048f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:06.840902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a606a5d-30af-43dd-8e5b-461ec04048f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:06.849466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:06.854813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a606a5d-30af-43dd-8e5b-461ec04048f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5a606a5d-30af-43dd-8e5b-461ec04048f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:06.861310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:06.862341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:06.863157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 256/490] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:06.863811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606a5d-30af-43dd-8e5b-461ec04048f1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:06.945728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d474af0-8d1a-4d28-936c-ca6bf8f5256f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:06.986423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61edd1-2e7c-416f-8635-7eec489daef8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:06.986833 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a606a5d-30af-43dd-8e5b-461ec04048f1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:06.987067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61edd1-2e7c-416f-8635-7eec489daef8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:06.987342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61edd1-2e7c-416f-8635-7eec489daef8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:06.987611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d61edd1-2e7c-416f-8635-7eec489daef8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.002221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61edd1-2e7c-416f-8635-7eec489daef8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.003981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d61edd1-2e7c-416f-8635-7eec489daef8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3d61edd1-2e7c-416f-8635-7eec489daef8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:07.022392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:07.023362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:07.024190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:07.024190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:07.024190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:07.024190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 235/491] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:07.024190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61edd1-2e7c-416f-8635-7eec489daef8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.027351 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e198a19-19ea-4abd-857d-bd5f74c43071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.028387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e198a19-19ea-4abd-857d-bd5f74c43071 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.028814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e198a19-19ea-4abd-857d-bd5f74c43071 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.029263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:07.029911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e198a19-19ea-4abd-857d-bd5f74c43071 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.030633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e198a19-19ea-4abd-857d-bd5f74c43071 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:07.030965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e198a19-19ea-4abd-857d-bd5f74c43071 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.032327 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e198a19-19ea-4abd-857d-bd5f74c43071 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.046109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 257/492] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:07.046109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77da34a-466c-4387-9bcb-824a992474ec None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.046109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77da34a-466c-4387-9bcb-824a992474ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.046109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77da34a-466c-4387-9bcb-824a992474ec None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.046109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 258/493] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:07.046109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77da34a-466c-4387-9bcb-824a992474ec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.106216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:07.150939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77da34a-466c-4387-9bcb-824a992474ec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:07.162467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d61edd1-2e7c-416f-8635-7eec489daef8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:07.188844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e3bab-4250-4ca0-86a6-eb8f0bca5608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.189439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e3bab-4250-4ca0-86a6-eb8f0bca5608 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.189710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e3bab-4250-4ca0-86a6-eb8f0bca5608 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.189994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691e3bab-4250-4ca0-86a6-eb8f0bca5608 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.190654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e3bab-4250-4ca0-86a6-eb8f0bca5608 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:07.190952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e3bab-4250-4ca0-86a6-eb8f0bca5608 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.192346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691e3bab-4250-4ca0-86a6-eb8f0bca5608 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.204255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 259/494] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:07.204991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e3bab-4250-4ca0-86a6-eb8f0bca5608 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.207569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2ffd24-9852-4bfa-84ee-3d7928342238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.208191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ffd24-9852-4bfa-84ee-3d7928342238 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.208473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ffd24-9852-4bfa-84ee-3d7928342238 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.208748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2ffd24-9852-4bfa-84ee-3d7928342238 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.209452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ffd24-9852-4bfa-84ee-3d7928342238 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:07.209753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ffd24-9852-4bfa-84ee-3d7928342238 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.211165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2ffd24-9852-4bfa-84ee-3d7928342238 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.220300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 236/495] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:07.221557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b04f8-60e5-4e95-8980-c53f1a7eb2ea None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.221845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b04f8-60e5-4e95-8980-c53f1a7eb2ea None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.222134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73b04f8-60e5-4e95-8980-c53f1a7eb2ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.240938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c73b04f8-60e5-4e95-8980-c53f1a7eb2ea None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:07.242153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c73b04f8-60e5-4e95-8980-c53f1a7eb2ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:07.303157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:07.311769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 237/496] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:07.312771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b04f8-60e5-4e95-8980-c53f1a7eb2ea None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.313556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691e3bab-4250-4ca0-86a6-eb8f0bca5608 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:07.334378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:07.351266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bee01-f5aa-48dd-bb3b-394cbc605ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.352309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bee01-f5aa-48dd-bb3b-394cbc605ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.352602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bee01-f5aa-48dd-bb3b-394cbc605ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.352876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bee01-f5aa-48dd-bb3b-394cbc605ca4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.364379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bee01-f5aa-48dd-bb3b-394cbc605ca4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.366343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bee01-f5aa-48dd-bb3b-394cbc605ca4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f71bee01-f5aa-48dd-bb3b-394cbc605ca4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:07.381266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:07.382228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:07.383079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:07.383079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:07.383079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:07.383079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 260/497] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:07.383963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bee01-f5aa-48dd-bb3b-394cbc605ca4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.422480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73b04f8-60e5-4e95-8980-c53f1a7eb2ea None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:07.459182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd5cdb1-dd03-41ec-940f-637cd10f2316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.459745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5cdb1-dd03-41ec-940f-637cd10f2316 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.460022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5cdb1-dd03-41ec-940f-637cd10f2316 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.460292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd5cdb1-dd03-41ec-940f-637cd10f2316 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.468010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5cdb1-dd03-41ec-940f-637cd10f2316 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.470253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd5cdb1-dd03-41ec-940f-637cd10f2316 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5cd5cdb1-dd03-41ec-940f-637cd10f2316 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:07.485492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:07.486432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:07.487367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:07.487888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bee01-f5aa-48dd-bb3b-394cbc605ca4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:07.487961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 238/498] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:07.488541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a7ac4-ac62-4b8a-829f-8fcd1a3d8d33 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.488541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a7ac4-ac62-4b8a-829f-8fcd1a3d8d33 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.488688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523a7ac4-ac62-4b8a-829f-8fcd1a3d8d33 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.494117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 239/499] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:07 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:07.494117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a7ac4-ac62-4b8a-829f-8fcd1a3d8d33 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.527968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a339a-64dc-4c58-a374-daf17d954ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.528601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a339a-64dc-4c58-a374-daf17d954ddf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.528877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a339a-64dc-4c58-a374-daf17d954ddf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.529157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028a339a-64dc-4c58-a374-daf17d954ddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.539899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a339a-64dc-4c58-a374-daf17d954ddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.541378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028a339a-64dc-4c58-a374-daf17d954ddf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-028a339a-64dc-4c58-a374-daf17d954ddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:07.556367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:07.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:07.558553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:07.558553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:07.558553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:07.558553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 261/500] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:07.558794 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a339a-64dc-4c58-a374-daf17d954ddf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.565820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:07.587102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:07.619554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523a7ac4-ac62-4b8a-829f-8fcd1a3d8d33 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:07.653108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a3805-d91e-4830-b1b7-8e89b7f6af96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.653889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a3805-d91e-4830-b1b7-8e89b7f6af96 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.654196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a3805-d91e-4830-b1b7-8e89b7f6af96 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.654470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a3805-d91e-4830-b1b7-8e89b7f6af96 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.655110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a3805-d91e-4830-b1b7-8e89b7f6af96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:07.655410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a3805-d91e-4830-b1b7-8e89b7f6af96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.656802 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a3805-d91e-4830-b1b7-8e89b7f6af96 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.662743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028a339a-64dc-4c58-a374-daf17d954ddf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:07.668252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 240/501] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:07.669688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c246867-badc-42d2-8215-deb3d5f8bfba None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.670540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c246867-badc-42d2-8215-deb3d5f8bfba None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.670818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c246867-badc-42d2-8215-deb3d5f8bfba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.691226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6c246867-badc-42d2-8215-deb3d5f8bfba None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:07.692439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6c246867-badc-42d2-8215-deb3d5f8bfba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:07.698489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb658c-303b-41d0-bb75-a953bca72ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.699097 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb658c-303b-41d0-bb75-a953bca72ebd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.699369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb658c-303b-41d0-bb75-a953bca72ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.699831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb658c-303b-41d0-bb75-a953bca72ebd None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.700726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb658c-303b-41d0-bb75-a953bca72ebd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:07.701030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb658c-303b-41d0-bb75-a953bca72ebd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.702492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6eb658c-303b-41d0-bb75-a953bca72ebd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.738186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 262/502] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:07.738934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb658c-303b-41d0-bb75-a953bca72ebd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.759808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 241/503] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:07.760645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c246867-badc-42d2-8215-deb3d5f8bfba None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.813722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:07.824323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:07.844517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb658c-303b-41d0-bb75-a953bca72ebd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:07.884075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3819e4-1991-4e27-82c2-03cab0851424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.884549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c246867-badc-42d2-8215-deb3d5f8bfba None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:07.884719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3819e4-1991-4e27-82c2-03cab0851424 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.884969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3819e4-1991-4e27-82c2-03cab0851424 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.885241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3819e4-1991-4e27-82c2-03cab0851424 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.885890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3819e4-1991-4e27-82c2-03cab0851424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:07.886223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3819e4-1991-4e27-82c2-03cab0851424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:07.887640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3819e4-1991-4e27-82c2-03cab0851424 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:07.906154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 263/504] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:07.906154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3819e4-1991-4e27-82c2-03cab0851424 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:07.934910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddb622-7f18-4a11-8c64-68694f9e80e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:07.935612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb622-7f18-4a11-8c64-68694f9e80e3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:07.935937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb622-7f18-4a11-8c64-68694f9e80e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:07.936273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ddb622-7f18-4a11-8c64-68694f9e80e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:07.948500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ddb622-7f18-4a11-8c64-68694f9e80e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b7ddb622-7f18-4a11-8c64-68694f9e80e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:07.953473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:07.954605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 242/505] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:07.955621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddb622-7f18-4a11-8c64-68694f9e80e3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.013665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3819e4-1991-4e27-82c2-03cab0851424 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.048102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ea2e5-5742-45f7-a385-19bfd400d245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.048735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea2e5-5742-45f7-a385-19bfd400d245 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.049012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea2e5-5742-45f7-a385-19bfd400d245 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.049286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ea2e5-5742-45f7-a385-19bfd400d245 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.056662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ea2e5-5742-45f7-a385-19bfd400d245 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.058500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ea2e5-5742-45f7-a385-19bfd400d245 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.064976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ddb622-7f18-4a11-8c64-68694f9e80e3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-957ea2e5-5742-45f7-a385-19bfd400d245 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:08.070869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:08.071798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:08.072675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 264/506] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:08.073645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ea2e5-5742-45f7-a385-19bfd400d245 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.076157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.078389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.101469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725af7c-634f-485b-bd35-52bab4b21f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.102112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725af7c-634f-485b-bd35-52bab4b21f48 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.102388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725af7c-634f-485b-bd35-52bab4b21f48 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.102662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5725af7c-634f-485b-bd35-52bab4b21f48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.168177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725af7c-634f-485b-bd35-52bab4b21f48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.169674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5725af7c-634f-485b-bd35-52bab4b21f48 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.175409 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ea2e5-5742-45f7-a385-19bfd400d245 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.214302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3e3c6-ccd5-48b3-9e52-6eeafe420047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.214930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3e3c6-ccd5-48b3-9e52-6eeafe420047 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.215227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3e3c6-ccd5-48b3-9e52-6eeafe420047 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.215496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c3e3c6-ccd5-48b3-9e52-6eeafe420047 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.222994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3e3c6-ccd5-48b3-9e52-6eeafe420047 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.224463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c3e3c6-ccd5-48b3-9e52-6eeafe420047 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d4c3e3c6-ccd5-48b3-9e52-6eeafe420047 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:08.235877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:08.236779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:08.237679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 265/507] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:08.238297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3e3c6-ccd5-48b3-9e52-6eeafe420047 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.275185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 243/508] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:08.275965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725af7c-634f-485b-bd35-52bab4b21f48 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.284403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.302728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.342599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c3e3c6-ccd5-48b3-9e52-6eeafe420047 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.374613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5725af7c-634f-485b-bd35-52bab4b21f48 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.376253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776358d3-3edd-4012-b27d-4f0bf1ac763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.376887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776358d3-3edd-4012-b27d-4f0bf1ac763f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.377163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776358d3-3edd-4012-b27d-4f0bf1ac763f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.377435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776358d3-3edd-4012-b27d-4f0bf1ac763f None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.378123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776358d3-3edd-4012-b27d-4f0bf1ac763f None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:08.378435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776358d3-3edd-4012-b27d-4f0bf1ac763f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.379841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-776358d3-3edd-4012-b27d-4f0bf1ac763f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.388347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 266/509] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:08.389188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776358d3-3edd-4012-b27d-4f0bf1ac763f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.409904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dde06-31ba-488a-9d07-472370696196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.410573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dde06-31ba-488a-9d07-472370696196 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.410856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dde06-31ba-488a-9d07-472370696196 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.411126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697dde06-31ba-488a-9d07-472370696196 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.421783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dde06-31ba-488a-9d07-472370696196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.423286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697dde06-31ba-488a-9d07-472370696196 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-697dde06-31ba-488a-9d07-472370696196 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:08.437754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:08.438853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:08.440079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:08.440079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:08.440079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:08.440079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 244/510] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:08.440079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dde06-31ba-488a-9d07-472370696196 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.491681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776358d3-3edd-4012-b27d-4f0bf1ac763f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.498556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.515049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.526291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd67f13-37cf-4bad-a457-a1cc0ee97835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.526821 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd67f13-37cf-4bad-a457-a1cc0ee97835 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.527100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd67f13-37cf-4bad-a457-a1cc0ee97835 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.527366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd67f13-37cf-4bad-a457-a1cc0ee97835 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.528778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd67f13-37cf-4bad-a457-a1cc0ee97835 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:08.529092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd67f13-37cf-4bad-a457-a1cc0ee97835 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.530575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd67f13-37cf-4bad-a457-a1cc0ee97835 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.542318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697dde06-31ba-488a-9d07-472370696196 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.548425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 267/511] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:08.549234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd67f13-37cf-4bad-a457-a1cc0ee97835 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.581063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc56e5-d97d-49a9-b60f-c1b4403520cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.581680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc56e5-d97d-49a9-b60f-c1b4403520cd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.581962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc56e5-d97d-49a9-b60f-c1b4403520cd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.582306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc56e5-d97d-49a9-b60f-c1b4403520cd None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.583050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc56e5-d97d-49a9-b60f-c1b4403520cd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:08.583383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc56e5-d97d-49a9-b60f-c1b4403520cd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.584816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc56e5-d97d-49a9-b60f-c1b4403520cd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.621877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 245/512] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:08.622762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc56e5-d97d-49a9-b60f-c1b4403520cd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.649529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd67f13-37cf-4bad-a457-a1cc0ee97835 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.684020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e486c-05e9-4192-87bd-10ace2bfef62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.684765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e486c-05e9-4192-87bd-10ace2bfef62 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.685046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e486c-05e9-4192-87bd-10ace2bfef62 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.685322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e486c-05e9-4192-87bd-10ace2bfef62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.695880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e486c-05e9-4192-87bd-10ace2bfef62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.697341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e486c-05e9-4192-87bd-10ace2bfef62 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-892e486c-05e9-4192-87bd-10ace2bfef62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:08.712134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:08.713052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:08.714116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:08.714116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:08.714116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:08.714116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 268/513] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:08.714782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e486c-05e9-4192-87bd-10ace2bfef62 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.730694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc56e5-d97d-49a9-b60f-c1b4403520cd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.747480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.749889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.766988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cf087-f991-4d37-b514-fcc332446cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.767627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cf087-f991-4d37-b514-fcc332446cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.767907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cf087-f991-4d37-b514-fcc332446cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.768183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35cf087-f991-4d37-b514-fcc332446cb6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.779513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cf087-f991-4d37-b514-fcc332446cb6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.780943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b35cf087-f991-4d37-b514-fcc332446cb6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b35cf087-f991-4d37-b514-fcc332446cb6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:08.796419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:08.797544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:08.798461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:08.798461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:08.798461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:08.798461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 246/514] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:08.798461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cf087-f991-4d37-b514-fcc332446cb6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.818900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e486c-05e9-4192-87bd-10ace2bfef62 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.851565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b7e1b-0960-4486-ad86-78efc6816be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.852205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b7e1b-0960-4486-ad86-78efc6816be7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.852426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b7e1b-0960-4486-ad86-78efc6816be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.852690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80b7e1b-0960-4486-ad86-78efc6816be7 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.853323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b7e1b-0960-4486-ad86-78efc6816be7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:08.853622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b7e1b-0960-4486-ad86-78efc6816be7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:08.855021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80b7e1b-0960-4486-ad86-78efc6816be7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:08.866855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 269/515] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:08.867638 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b7e1b-0960-4486-ad86-78efc6816be7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:08.899716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b35cf087-f991-4d37-b514-fcc332446cb6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.933420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcc6cd-5830-4990-a13e-0e01301941e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:08.934179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcc6cd-5830-4990-a13e-0e01301941e3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:08.934581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcc6cd-5830-4990-a13e-0e01301941e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:08.934863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbcc6cd-5830-4990-a13e-0e01301941e3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:08.959438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.966879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80b7e1b-0960-4486-ad86-78efc6816be7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:08.968490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:08.999917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa7b3f-4c3d-4f8e-ab23-d6a636a6e69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.000523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa7b3f-4c3d-4f8e-ab23-d6a636a6e69d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.000798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa7b3f-4c3d-4f8e-ab23-d6a636a6e69d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.001069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aa7b3f-4c3d-4f8e-ab23-d6a636a6e69d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.004568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcc6cd-5830-4990-a13e-0e01301941e3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.006051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbcc6cd-5830-4990-a13e-0e01301941e3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.075710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa7b3f-4c3d-4f8e-ab23-d6a636a6e69d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.077094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aa7b3f-4c3d-4f8e-ab23-d6a636a6e69d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.110276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 247/516] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:09.111146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcc6cd-5830-4990-a13e-0e01301941e3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.168181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:09.175108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:09.176209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 270/517] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:09.177024 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa7b3f-4c3d-4f8e-ab23-d6a636a6e69d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.214936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbcc6cd-5830-4990-a13e-0e01301941e3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.250706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535427d1-078c-4a02-a163-74958725f271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.251317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535427d1-078c-4a02-a163-74958725f271 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.251597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535427d1-078c-4a02-a163-74958725f271 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.251869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535427d1-078c-4a02-a163-74958725f271 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.252508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535427d1-078c-4a02-a163-74958725f271 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:09.252808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535427d1-078c-4a02-a163-74958725f271 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.254240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535427d1-078c-4a02-a163-74958725f271 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.265580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 248/518] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:09.266396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535427d1-078c-4a02-a163-74958725f271 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.278937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aa7b3f-4c3d-4f8e-ab23-d6a636a6e69d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.314558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a20a9-7ad5-4b06-a83c-a2ca3de3beb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.315171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a20a9-7ad5-4b06-a83c-a2ca3de3beb1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.315449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a20a9-7ad5-4b06-a83c-a2ca3de3beb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.315725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109a20a9-7ad5-4b06-a83c-a2ca3de3beb1 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.316369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a20a9-7ad5-4b06-a83c-a2ca3de3beb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:09.316667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a20a9-7ad5-4b06-a83c-a2ca3de3beb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.318105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109a20a9-7ad5-4b06-a83c-a2ca3de3beb1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.329851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 271/519] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:09.330624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a20a9-7ad5-4b06-a83c-a2ca3de3beb1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.369299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535427d1-078c-4a02-a163-74958725f271 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.385619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:09.396048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:09.404041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102bbdd-2372-4173-961a-7bba612fa379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.404651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102bbdd-2372-4173-961a-7bba612fa379 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.404934 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102bbdd-2372-4173-961a-7bba612fa379 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.405205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1102bbdd-2372-4173-961a-7bba612fa379 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.405851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102bbdd-2372-4173-961a-7bba612fa379 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.407382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1102bbdd-2372-4173-961a-7bba612fa379 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.430623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 249/520] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:48:09 2022] POST /identity/v3/users => generated 247 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:09.431435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109a20a9-7ad5-4b06-a83c-a2ca3de3beb1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.431543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102bbdd-2372-4173-961a-7bba612fa379 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.468818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873eff5-7eea-492b-94f1-a0be5a8b0430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.469522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873eff5-7eea-492b-94f1-a0be5a8b0430 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.469799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873eff5-7eea-492b-94f1-a0be5a8b0430 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.470074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4873eff5-7eea-492b-94f1-a0be5a8b0430 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.481606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873eff5-7eea-492b-94f1-a0be5a8b0430 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.483263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4873eff5-7eea-492b-94f1-a0be5a8b0430 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4873eff5-7eea-492b-94f1-a0be5a8b0430 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:09.499609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:09.500644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:09.501518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:09.501518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:09.501518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:09.501518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 272/521] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:09.502442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873eff5-7eea-492b-94f1-a0be5a8b0430 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.566899 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1102bbdd-2372-4173-961a-7bba612fa379 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.606444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23fbe9-bce0-4a22-89c9-75551002ee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.606659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23fbe9-bce0-4a22-89c9-75551002ee9b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.606910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23fbe9-bce0-4a22-89c9-75551002ee9b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.607237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb23fbe9-bce0-4a22-89c9-75551002ee9b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.622287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23fbe9-bce0-4a22-89c9-75551002ee9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.624582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb23fbe9-bce0-4a22-89c9-75551002ee9b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.624730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4873eff5-7eea-492b-94f1-a0be5a8b0430 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.631669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-cb23fbe9-bce0-4a22-89c9-75551002ee9b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:09.640435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:09.641386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:09.642283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:09.642283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:09.642283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:09.642283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 250/522] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:09.642941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23fbe9-bce0-4a22-89c9-75551002ee9b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.670273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d8136-720b-425b-b5bb-8606dfeac097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.670917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d8136-720b-425b-b5bb-8606dfeac097 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.671201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d8136-720b-425b-b5bb-8606dfeac097 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.671476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70d8136-720b-425b-b5bb-8606dfeac097 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.679267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d8136-720b-425b-b5bb-8606dfeac097 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.680741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70d8136-720b-425b-b5bb-8606dfeac097 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.682463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f70d8136-720b-425b-b5bb-8606dfeac097 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:09.693872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:09.695029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:09.695879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:09.695879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:09.695879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:09.695879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:09.695879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:09.695879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:09.695879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:09.695879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:09.695879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 273/523] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:09.696608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d8136-720b-425b-b5bb-8606dfeac097 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.761758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb23fbe9-bce0-4a22-89c9-75551002ee9b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.800213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f3092-6038-45f1-909a-32fc9039bccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.800460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70d8136-720b-425b-b5bb-8606dfeac097 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.800869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f3092-6038-45f1-909a-32fc9039bccd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.801151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f3092-6038-45f1-909a-32fc9039bccd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.801422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605f3092-6038-45f1-909a-32fc9039bccd None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.809579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f3092-6038-45f1-909a-32fc9039bccd None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.811236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605f3092-6038-45f1-909a-32fc9039bccd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-605f3092-6038-45f1-909a-32fc9039bccd None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:09.823175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:09.824198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:09.825223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 251/524] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/users/neutron => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:09.825871 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f3092-6038-45f1-909a-32fc9039bccd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.843538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d508e-90c4-4bc7-bb5b-7286bc69274b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.844187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d508e-90c4-4bc7-bb5b-7286bc69274b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.844466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d508e-90c4-4bc7-bb5b-7286bc69274b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.844745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d508e-90c4-4bc7-bb5b-7286bc69274b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.845396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d508e-90c4-4bc7-bb5b-7286bc69274b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:09.845687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d508e-90c4-4bc7-bb5b-7286bc69274b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.847139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4d508e-90c4-4bc7-bb5b-7286bc69274b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.863925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 274/525] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:09.863925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d508e-90c4-4bc7-bb5b-7286bc69274b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:09.889779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:09.932657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:09.939816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605f3092-6038-45f1-909a-32fc9039bccd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.970321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d508e-90c4-4bc7-bb5b-7286bc69274b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:09.979255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b19368-723b-40ca-87ab-67c2ee578a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:09.980079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b19368-723b-40ca-87ab-67c2ee578a1e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:09.980518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b19368-723b-40ca-87ab-67c2ee578a1e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:09.980814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b19368-723b-40ca-87ab-67c2ee578a1e None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:09.981513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b19368-723b-40ca-87ab-67c2ee578a1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:09.981821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b19368-723b-40ca-87ab-67c2ee578a1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:09.983341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b19368-723b-40ca-87ab-67c2ee578a1e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:09.995776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 252/526] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:09.996627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b19368-723b-40ca-87ab-67c2ee578a1e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.010068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b1542-ad0d-453d-9e65-1bf43b1514b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.010716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b1542-ad0d-453d-9e65-1bf43b1514b9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.010996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b1542-ad0d-453d-9e65-1bf43b1514b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.011262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b1542-ad0d-453d-9e65-1bf43b1514b9 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.011958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b1542-ad0d-453d-9e65-1bf43b1514b9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:10.012342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b1542-ad0d-453d-9e65-1bf43b1514b9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.013806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b1542-ad0d-453d-9e65-1bf43b1514b9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.031961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 275/527] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:10.032792 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b1542-ad0d-453d-9e65-1bf43b1514b9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.110707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b19368-723b-40ca-87ab-67c2ee578a1e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:10.119615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:10.137420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b1542-ad0d-453d-9e65-1bf43b1514b9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:10.148662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c30464-9b72-40e6-b0db-e0ef61e10690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.149393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c30464-9b72-40e6-b0db-e0ef61e10690 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.149598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c30464-9b72-40e6-b0db-e0ef61e10690 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.149808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c30464-9b72-40e6-b0db-e0ef61e10690 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.150581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c30464-9b72-40e6-b0db-e0ef61e10690 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:10.150878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c30464-9b72-40e6-b0db-e0ef61e10690 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.152325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c30464-9b72-40e6-b0db-e0ef61e10690 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.172343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 253/528] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Sep 13 06:48:09 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:10.172983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c30464-9b72-40e6-b0db-e0ef61e10690 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.177337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835df90-46d3-4d10-8d49-0359412cf494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.178289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835df90-46d3-4d10-8d49-0359412cf494 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.178770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835df90-46d3-4d10-8d49-0359412cf494 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.179235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0835df90-46d3-4d10-8d49-0359412cf494 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.189181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:10.190723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835df90-46d3-4d10-8d49-0359412cf494 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.192907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0835df90-46d3-4d10-8d49-0359412cf494 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0835df90-46d3-4d10-8d49-0359412cf494 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:10.207780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:10.208681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:10.209530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:10.209530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:10.209530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:10.209530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:10.209530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:10.209530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:10.209530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:10.209530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:10.212247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 276/529] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:10.213155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835df90-46d3-4d10-8d49-0359412cf494 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.280553 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c30464-9b72-40e6-b0db-e0ef61e10690 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:10.317158 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ce12f-b610-439c-bb96-ad42cbf18db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.318050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ce12f-b610-439c-bb96-ad42cbf18db1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.318366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ce12f-b610-439c-bb96-ad42cbf18db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.318640 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513ce12f-b610-439c-bb96-ad42cbf18db1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.320407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0835df90-46d3-4d10-8d49-0359412cf494 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:10.325865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ce12f-b610-439c-bb96-ad42cbf18db1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.327684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513ce12f-b610-439c-bb96-ad42cbf18db1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-513ce12f-b610-439c-bb96-ad42cbf18db1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:10.341398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:10.342440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:10.343282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:10.343282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:10.343282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:10.343282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:10.343282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:10.343282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:10.343282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:10.343282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:10.343706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 254/530] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:10.344176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ce12f-b610-439c-bb96-ad42cbf18db1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.358474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc84c05-f0d1-4402-b25a-87d03fbb2204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.359428 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc84c05-f0d1-4402-b25a-87d03fbb2204 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.359723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc84c05-f0d1-4402-b25a-87d03fbb2204 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.360008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc84c05-f0d1-4402-b25a-87d03fbb2204 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.360760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc84c05-f0d1-4402-b25a-87d03fbb2204 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=87f3a9ffb88f4faa85edc3dabce0c8f5) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:10.361077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc84c05-f0d1-4402-b25a-87d03fbb2204 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.362602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc84c05-f0d1-4402-b25a-87d03fbb2204 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.374199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 277/531] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:10 2022] GET /identity/v3/role_assignments?user.id=87f3a9ffb88f4faa85edc3dabce0c8f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:10.374199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc84c05-f0d1-4402-b25a-87d03fbb2204 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.393918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:10.439820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:10.463528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513ce12f-b610-439c-bb96-ad42cbf18db1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:10.501844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e81eae-20bf-4199-8cad-e7b0ce94dcbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.502600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e81eae-20bf-4199-8cad-e7b0ce94dcbd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.502870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e81eae-20bf-4199-8cad-e7b0ce94dcbd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.503231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e81eae-20bf-4199-8cad-e7b0ce94dcbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.510355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc84c05-f0d1-4402-b25a-87d03fbb2204 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:10.561573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1b1f1-8e55-4022-9cbb-41f50148d8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.562651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b1f1-8e55-4022-9cbb-41f50148d8e1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.563448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b1f1-8e55-4022-9cbb-41f50148d8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.563822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d1b1f1-8e55-4022-9cbb-41f50148d8e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.564642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1b1f1-8e55-4022-9cbb-41f50148d8e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:10.565053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1b1f1-8e55-4022-9cbb-41f50148d8e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.566904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d1b1f1-8e55-4022-9cbb-41f50148d8e1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.588663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 278/532] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:10.589557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1b1f1-8e55-4022-9cbb-41f50148d8e1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.591161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e81eae-20bf-4199-8cad-e7b0ce94dcbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.593653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e81eae-20bf-4199-8cad-e7b0ce94dcbd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.678054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:10.692207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:10.716006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d1b1f1-8e55-4022-9cbb-41f50148d8e1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:10.722806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 255/533] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:10.723739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e81eae-20bf-4199-8cad-e7b0ce94dcbd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.763365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b14e6-0858-49a8-a38a-e0b6f1a62ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.764255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b14e6-0858-49a8-a38a-e0b6f1a62ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.765441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b14e6-0858-49a8-a38a-e0b6f1a62ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.765721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360b14e6-0858-49a8-a38a-e0b6f1a62ec8 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.766464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b14e6-0858-49a8-a38a-e0b6f1a62ec8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:10.766770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b14e6-0858-49a8-a38a-e0b6f1a62ec8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.768226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360b14e6-0858-49a8-a38a-e0b6f1a62ec8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.783988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 279/534] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:10.785376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b14e6-0858-49a8-a38a-e0b6f1a62ec8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.835324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e81eae-20bf-4199-8cad-e7b0ce94dcbd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:10.873593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e50d4-0a02-4ad0-9f00-1c73675b5e63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.874259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e50d4-0a02-4ad0-9f00-1c73675b5e63 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.874541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e50d4-0a02-4ad0-9f00-1c73675b5e63 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.874813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8e50d4-0a02-4ad0-9f00-1c73675b5e63 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.875561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e50d4-0a02-4ad0-9f00-1c73675b5e63 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=3505c8965c4a42e493a2f5916cab56dc) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:10.875874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e50d4-0a02-4ad0-9f00-1c73675b5e63 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.877337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8e50d4-0a02-4ad0-9f00-1c73675b5e63 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.883199 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 256/535] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:10 2022] GET /identity/v3/role_assignments?user.id=3505c8965c4a42e493a2f5916cab56dc&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:10.884027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e50d4-0a02-4ad0-9f00-1c73675b5e63 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:10.920083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360b14e6-0858-49a8-a38a-e0b6f1a62ec8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:10.929468 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:10.957754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:10.961434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cb220-99bf-47a9-b7d7-35f02bef82cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:10.962103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb220-99bf-47a9-b7d7-35f02bef82cc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:10.962442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb220-99bf-47a9-b7d7-35f02bef82cc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:10.962714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9cb220-99bf-47a9-b7d7-35f02bef82cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:10.975485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cb220-99bf-47a9-b7d7-35f02bef82cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:10.977134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9cb220-99bf-47a9-b7d7-35f02bef82cc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:10.996657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8e50d4-0a02-4ad0-9f00-1c73675b5e63 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7b9cb220-99bf-47a9-b7d7-35f02bef82cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:11.005438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:11.007588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:11.009614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:11.009614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:11.009614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:11.015022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 280/536] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:11.021091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cb220-99bf-47a9-b7d7-35f02bef82cc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:11.055343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da560c-aebe-47df-b3de-113a3cbbfc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:11.055981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da560c-aebe-47df-b3de-113a3cbbfc2d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:11.056258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da560c-aebe-47df-b3de-113a3cbbfc2d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:11.056532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96da560c-aebe-47df-b3de-113a3cbbfc2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:11.057298 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da560c-aebe-47df-b3de-113a3cbbfc2d None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=88520ce6493a44f39b19c67592f03632) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:11.057610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da560c-aebe-47df-b3de-113a3cbbfc2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:11.059603 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96da560c-aebe-47df-b3de-113a3cbbfc2d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:11.072800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 257/537] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:10 2022] GET /identity/v3/role_assignments?user.id=88520ce6493a44f39b19c67592f03632&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:11.186608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9cb220-99bf-47a9-b7d7-35f02bef82cc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:11.239117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243597db-6215-4b3a-a47d-47d450852c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:11.239764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243597db-6215-4b3a-a47d-47d450852c02 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:11.240046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243597db-6215-4b3a-a47d-47d450852c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:11.240319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243597db-6215-4b3a-a47d-47d450852c02 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:11.240983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243597db-6215-4b3a-a47d-47d450852c02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:11.241284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243597db-6215-4b3a-a47d-47d450852c02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:11.242729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243597db-6215-4b3a-a47d-47d450852c02 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:11.257369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 281/538] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:11.263381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da560c-aebe-47df-b3de-113a3cbbfc2d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:11.414598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96da560c-aebe-47df-b3de-113a3cbbfc2d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:11.422774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:11.461008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24363626-be77-48d7-b097-051086a1b9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:11.461563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24363626-be77-48d7-b097-051086a1b9de None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:11.461836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24363626-be77-48d7-b097-051086a1b9de None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:11.462121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24363626-be77-48d7-b097-051086a1b9de None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:11.469058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24363626-be77-48d7-b097-051086a1b9de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:11.470550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24363626-be77-48d7-b097-051086a1b9de None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-24363626-be77-48d7-b097-051086a1b9de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:11.481843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:11.483604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:11.484704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:11.484704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:11.484704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:11.484704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:11.484704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:11.484704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:11.484704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:11.484704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:11.484704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 258/539] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:11.491216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243597db-6215-4b3a-a47d-47d450852c02 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:11.555536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:11.632396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243597db-6215-4b3a-a47d-47d450852c02 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:11.684920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98849b86-7ddc-4d5f-99b1-ba4c961ab72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:11.685662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98849b86-7ddc-4d5f-99b1-ba4c961ab72c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:11.685943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98849b86-7ddc-4d5f-99b1-ba4c961ab72c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:11.686255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98849b86-7ddc-4d5f-99b1-ba4c961ab72c None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:11.686962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98849b86-7ddc-4d5f-99b1-ba4c961ab72c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:11.687313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98849b86-7ddc-4d5f-99b1-ba4c961ab72c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:11.688790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98849b86-7ddc-4d5f-99b1-ba4c961ab72c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:11.705561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 282/540] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:11.711584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24363626-be77-48d7-b097-051086a1b9de None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:11.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6820c5b-b8ee-462e-9a63-611dad48f801 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:11.741915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6820c5b-b8ee-462e-9a63-611dad48f801 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:11.742432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6820c5b-b8ee-462e-9a63-611dad48f801 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:11.744209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 283/541] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:11.749682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a8ac3-f5a5-4f09-a267-68dd63430d29 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:11.750278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a8ac3-f5a5-4f09-a267-68dd63430d29 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:11.750770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777a8ac3-f5a5-4f09-a267-68dd63430d29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:11.781587 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-777a8ac3-f5a5-4f09-a267-68dd63430d29 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:11.783147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-777a8ac3-f5a5-4f09-a267-68dd63430d29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:11.889820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24363626-be77-48d7-b097-051086a1b9de None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:11.900378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 284/542] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:11.939734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1411b5-3a2d-45c7-9f25-182de9822162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:11.940717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1411b5-3a2d-45c7-9f25-182de9822162 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:11.941192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1411b5-3a2d-45c7-9f25-182de9822162 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:11.941699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1411b5-3a2d-45c7-9f25-182de9822162 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:11.942687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1411b5-3a2d-45c7-9f25-182de9822162 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=f94aa8af227a42a08d8a0e4b7f02b85a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:11.943193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1411b5-3a2d-45c7-9f25-182de9822162 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:11.944809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1411b5-3a2d-45c7-9f25-182de9822162 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:11.951281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 259/543] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:11 2022] GET /identity/v3/role_assignments?user.id=f94aa8af227a42a08d8a0e4b7f02b85a&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:12.305108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a8ac3-f5a5-4f09-a267-68dd63430d29 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:12.338714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:12.426823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777a8ac3-f5a5-4f09-a267-68dd63430d29 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:12.473521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3484b-b1ed-489b-a37d-da4f25e34f15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:12.474570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3484b-b1ed-489b-a37d-da4f25e34f15 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:12.475045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3484b-b1ed-489b-a37d-da4f25e34f15 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:12.475531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc3484b-b1ed-489b-a37d-da4f25e34f15 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:12.495039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc3484b-b1ed-489b-a37d-da4f25e34f15 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ebc3484b-b1ed-489b-a37d-da4f25e34f15 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:12.499974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:12.500884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:12.501677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 285/544] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:12.508816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1411b5-3a2d-45c7-9f25-182de9822162 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:12.552931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:12.634648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1411b5-3a2d-45c7-9f25-182de9822162 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:12.678880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77447d27-cc07-4492-b9eb-eda6978b61d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:12.679501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77447d27-cc07-4492-b9eb-eda6978b61d3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:12.679783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77447d27-cc07-4492-b9eb-eda6978b61d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:12.680057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77447d27-cc07-4492-b9eb-eda6978b61d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:12.680719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77447d27-cc07-4492-b9eb-eda6978b61d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:12.681020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77447d27-cc07-4492-b9eb-eda6978b61d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:12.682420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77447d27-cc07-4492-b9eb-eda6978b61d3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:12.697398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 260/545] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:12.703416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3484b-b1ed-489b-a37d-da4f25e34f15 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:12.829292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:12.831784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93df56f-bc7d-4f91-b2cf-702c3da209b1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:12.832072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93df56f-bc7d-4f91-b2cf-702c3da209b1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:12.832359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93df56f-bc7d-4f91-b2cf-702c3da209b1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:12.833999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 261/546] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:12.838398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc3484b-b1ed-489b-a37d-da4f25e34f15 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:12.839101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780b605-6f61-4481-9256-8752cd789b84 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:12.839382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780b605-6f61-4481-9256-8752cd789b84 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:12.839657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e780b605-6f61-4481-9256-8752cd789b84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:12.858151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e780b605-6f61-4481-9256-8752cd789b84 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:12.859414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e780b605-6f61-4481-9256-8752cd789b84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:12.885270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724c133-6a64-4bd0-862d-3515a8643a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:12.885924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724c133-6a64-4bd0-862d-3515a8643a7f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:12.886231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724c133-6a64-4bd0-862d-3515a8643a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:12.886504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3724c133-6a64-4bd0-862d-3515a8643a7f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:12.900889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724c133-6a64-4bd0-862d-3515a8643a7f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:12.902404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3724c133-6a64-4bd0-862d-3515a8643a7f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3724c133-6a64-4bd0-862d-3515a8643a7f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:12.930762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:12.931739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:12.932589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:12.932589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:12.932589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:12.932589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 286/547] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:12.939244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724c133-6a64-4bd0-862d-3515a8643a7f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:12.965150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 262/548] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:48:13.061223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3724c133-6a64-4bd0-862d-3515a8643a7f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:13.100805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2233b-b456-44d1-adb1-d657a8544987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:13.101424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2233b-b456-44d1-adb1-d657a8544987 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:13.101703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2233b-b456-44d1-adb1-d657a8544987 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:13.101977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2233b-b456-44d1-adb1-d657a8544987 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:13.102652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2233b-b456-44d1-adb1-d657a8544987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:13.102955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2233b-b456-44d1-adb1-d657a8544987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:13.104347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2233b-b456-44d1-adb1-d657a8544987 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:13.117826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 287/549] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:13.123371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780b605-6f61-4481-9256-8752cd789b84 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:13.148931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:13.258313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e780b605-6f61-4481-9256-8752cd789b84 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:13.283890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2233b-b456-44d1-adb1-d657a8544987 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:13.288036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:13.319572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821367dc-c020-4986-a2fa-7bf6a30cadff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:13.320223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821367dc-c020-4986-a2fa-7bf6a30cadff None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:13.320502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821367dc-c020-4986-a2fa-7bf6a30cadff None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:13.320771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821367dc-c020-4986-a2fa-7bf6a30cadff None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:13.338124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821367dc-c020-4986-a2fa-7bf6a30cadff None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:13.339597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821367dc-c020-4986-a2fa-7bf6a30cadff None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-821367dc-c020-4986-a2fa-7bf6a30cadff None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:13.363746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:13.364991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:13.365921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 263/550] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:13 2022] GET /identity/v3/users/swift => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:13.372032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821367dc-c020-4986-a2fa-7bf6a30cadff None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:13.396958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2233b-b456-44d1-adb1-d657a8544987 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:13.440649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b7289-5d2d-49b2-b8c0-66a5b094ba93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:13.441362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b7289-5d2d-49b2-b8c0-66a5b094ba93 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:13.441641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b7289-5d2d-49b2-b8c0-66a5b094ba93 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:13.441912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3b7289-5d2d-49b2-b8c0-66a5b094ba93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:13.463315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3b7289-5d2d-49b2-b8c0-66a5b094ba93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-aa3b7289-5d2d-49b2-b8c0-66a5b094ba93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:13.468887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:13.469764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:13.470824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:13.483297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 288/551] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:13.485617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b7289-5d2d-49b2-b8c0-66a5b094ba93 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:13.494491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:13.531649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821367dc-c020-4986-a2fa-7bf6a30cadff None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:13.584380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:13.625193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e302ab-af88-4dfc-ae4b-abe4a6743d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:13.625740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e302ab-af88-4dfc-ae4b-abe4a6743d72 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:13.626015 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e302ab-af88-4dfc-ae4b-abe4a6743d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:13.626310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e302ab-af88-4dfc-ae4b-abe4a6743d72 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:13.627048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e302ab-af88-4dfc-ae4b-abe4a6743d72 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:13.627355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e302ab-af88-4dfc-ae4b-abe4a6743d72 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:13.628803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e302ab-af88-4dfc-ae4b-abe4a6743d72 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:13.662219 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 264/552] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Sep 13 06:48:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:13.664846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3b7289-5d2d-49b2-b8c0-66a5b094ba93 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:13.668102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e302ab-af88-4dfc-ae4b-abe4a6743d72 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:13.745686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c602a6-bdc6-4168-9fb2-3611d7c708cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:13.746364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c602a6-bdc6-4168-9fb2-3611d7c708cb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:13.746662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c602a6-bdc6-4168-9fb2-3611d7c708cb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:13.746935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c602a6-bdc6-4168-9fb2-3611d7c708cb None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:13.747601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c602a6-bdc6-4168-9fb2-3611d7c708cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:13.747903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c602a6-bdc6-4168-9fb2-3611d7c708cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:13.749294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c602a6-bdc6-4168-9fb2-3611d7c708cb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:13.770425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 289/553] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:13.779195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c602a6-bdc6-4168-9fb2-3611d7c708cb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:13.857531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e302ab-af88-4dfc-ae4b-abe4a6743d72 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:13.892132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c602a6-bdc6-4168-9fb2-3611d7c708cb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:13.893091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38418fd6-9240-49da-bc7c-606a540da2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:13.893813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418fd6-9240-49da-bc7c-606a540da2bb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:13.894110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418fd6-9240-49da-bc7c-606a540da2bb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:13.894956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38418fd6-9240-49da-bc7c-606a540da2bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:13.897730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:13.924657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:13.936527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0b363d-d854-4bdd-ac68-96981a3a5212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:13.937163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b363d-d854-4bdd-ac68-96981a3a5212 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:13.937846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b363d-d854-4bdd-ac68-96981a3a5212 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:13.938112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0b363d-d854-4bdd-ac68-96981a3a5212 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:13.949923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b363d-d854-4bdd-ac68-96981a3a5212 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:13.951438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0b363d-d854-4bdd-ac68-96981a3a5212 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-af0b363d-d854-4bdd-ac68-96981a3a5212 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:13.966780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:13.967862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:13.968945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:13.968945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:13.968945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:13.969861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418fd6-9240-49da-bc7c-606a540da2bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:13.971337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38418fd6-9240-49da-bc7c-606a540da2bb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:13.972660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 290/554] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:13.974189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc80c18-54bc-4d4e-92cf-06e788a15bec None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:13.974189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc80c18-54bc-4d4e-92cf-06e788a15bec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:13.974470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc80c18-54bc-4d4e-92cf-06e788a15bec None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:13.975732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 291/555] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:13.976739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4e8ab-0adc-46c9-89bf-4cc927d0a558 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:13.977017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4e8ab-0adc-46c9-89bf-4cc927d0a558 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:13.977292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4e8ab-0adc-46c9-89bf-4cc927d0a558 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:13.978482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 292/556] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:13.979172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4e8ab-0adc-46c9-89bf-4cc927d0a558 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:14.081274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 265/557] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 13 06:48:14.082695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e503d4a-28f9-4c46-9db8-2bd28f0d0cf9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.083048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e503d4a-28f9-4c46-9db8-2bd28f0d0cf9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.083389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e503d4a-28f9-4c46-9db8-2bd28f0d0cf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.097370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e4e8ab-0adc-46c9-89bf-4cc927d0a558 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:14.103316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7e503d4a-28f9-4c46-9db8-2bd28f0d0cf9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:14.104666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7e503d4a-28f9-4c46-9db8-2bd28f0d0cf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:14.130143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbd4a42-c733-4e5d-a51b-2718739ebf0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:14.130791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd4a42-c733-4e5d-a51b-2718739ebf0e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.131025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd4a42-c733-4e5d-a51b-2718739ebf0e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.131439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbd4a42-c733-4e5d-a51b-2718739ebf0e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.132079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd4a42-c733-4e5d-a51b-2718739ebf0e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:14.132379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd4a42-c733-4e5d-a51b-2718739ebf0e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:14.133755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbd4a42-c733-4e5d-a51b-2718739ebf0e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:14.138197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:14.150767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 293/558] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:14.150767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804f8c0-7569-47ad-80dd-f223b9a5d58d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.150767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804f8c0-7569-47ad-80dd-f223b9a5d58d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.150767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6804f8c0-7569-47ad-80dd-f223b9a5d58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.167472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6804f8c0-7569-47ad-80dd-f223b9a5d58d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:14.168673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6804f8c0-7569-47ad-80dd-f223b9a5d58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:14.198104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 266/559] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:14.198104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e503d4a-28f9-4c46-9db8-2bd28f0d0cf9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:14.213473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:14.266828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 294/560] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:14.268119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804f8c0-7569-47ad-80dd-f223b9a5d58d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:14.354172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e503d4a-28f9-4c46-9db8-2bd28f0d0cf9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:14.428038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834d786-230a-49a7-ae13-ea3c8bd4d1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:14.429847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834d786-230a-49a7-ae13-ea3c8bd4d1a2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.431151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834d786-230a-49a7-ae13-ea3c8bd4d1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.432776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6834d786-230a-49a7-ae13-ea3c8bd4d1a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.444632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6804f8c0-7569-47ad-80dd-f223b9a5d58d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:14.456317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834d786-230a-49a7-ae13-ea3c8bd4d1a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:14.458032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6834d786-230a-49a7-ae13-ea3c8bd4d1a2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6834d786-230a-49a7-ae13-ea3c8bd4d1a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:14.487446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:14.488448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:14.489318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:14.489318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:14.489318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:14.489318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 267/561] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:14.490016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834d786-230a-49a7-ae13-ea3c8bd4d1a2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:14.505758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62372d-4973-4d39-8094-d1be27b34cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:14.507016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62372d-4973-4d39-8094-d1be27b34cbe None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.508015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62372d-4973-4d39-8094-d1be27b34cbe None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.508624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62372d-4973-4d39-8094-d1be27b34cbe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.511563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:14.519779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62372d-4973-4d39-8094-d1be27b34cbe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:14.522048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62372d-4973-4d39-8094-d1be27b34cbe None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1c62372d-4973-4d39-8094-d1be27b34cbe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:14.538506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:14.539486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:14.540412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:14.542847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 295/562] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Sep 13 06:48:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:14.544052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62372d-4973-4d39-8094-d1be27b34cbe None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:14.583562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:14.619425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6834d786-230a-49a7-ae13-ea3c8bd4d1a2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:14.659632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb020f-2d87-4fe2-87da-9d76f2b73ec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:14.660299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb020f-2d87-4fe2-87da-9d76f2b73ec3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.660582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb020f-2d87-4fe2-87da-9d76f2b73ec3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.660884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cb020f-2d87-4fe2-87da-9d76f2b73ec3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.667909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62372d-4973-4d39-8094-d1be27b34cbe None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:14.672369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cb020f-2d87-4fe2-87da-9d76f2b73ec3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b8cb020f-2d87-4fe2-87da-9d76f2b73ec3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:14.677751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:14.682334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 268/563] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:14.683514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb020f-2d87-4fe2-87da-9d76f2b73ec3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:14.707816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b0fce-a605-4ba0-b358-97c7f84f51fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:14.708439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b0fce-a605-4ba0-b358-97c7f84f51fd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.708720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b0fce-a605-4ba0-b358-97c7f84f51fd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.708994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b0fce-a605-4ba0-b358-97c7f84f51fd None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.709912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b0fce-a605-4ba0-b358-97c7f84f51fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:14.710021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b0fce-a605-4ba0-b358-97c7f84f51fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:14.711387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b0fce-a605-4ba0-b358-97c7f84f51fd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:14.724159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 296/564] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:14.724937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b0fce-a605-4ba0-b358-97c7f84f51fd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:14.760566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:14.782194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cb020f-2d87-4fe2-87da-9d76f2b73ec3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:14.813524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:14.816396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5f5905-8453-4e52-8ad9-7e803395325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:14.817055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5905-8453-4e52-8ad9-7e803395325c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.817333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5905-8453-4e52-8ad9-7e803395325c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.817599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5f5905-8453-4e52-8ad9-7e803395325c None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.818359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f5905-8453-4e52-8ad9-7e803395325c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:14.818669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f5905-8453-4e52-8ad9-7e803395325c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:14.820110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5f5905-8453-4e52-8ad9-7e803395325c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:14.826340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b0fce-a605-4ba0-b358-97c7f84f51fd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:14.837501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 269/565] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Sep 13 06:48:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:14.838749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe097d9-c358-4ebe-8386-25ddaf967cf5 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.839036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe097d9-c358-4ebe-8386-25ddaf967cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.839311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe097d9-c358-4ebe-8386-25ddaf967cf5 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.840573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 270/566] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:14.841263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe097d9-c358-4ebe-8386-25ddaf967cf5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:14.865722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d95d8b-2e33-48e6-a4b1-2b6b780aa4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:14.866392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d95d8b-2e33-48e6-a4b1-2b6b780aa4db None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.866673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d95d8b-2e33-48e6-a4b1-2b6b780aa4db None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.866951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d95d8b-2e33-48e6-a4b1-2b6b780aa4db None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.877122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d95d8b-2e33-48e6-a4b1-2b6b780aa4db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c3d95d8b-2e33-48e6-a4b1-2b6b780aa4db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:14.881950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:14.882898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:14.883726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 297/567] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:14.884457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d95d8b-2e33-48e6-a4b1-2b6b780aa4db None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:14.945875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe097d9-c358-4ebe-8386-25ddaf967cf5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:14.986185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f09cbd-095e-4fd5-8b5f-4ad82324ee8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:14.986790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f09cbd-095e-4fd5-8b5f-4ad82324ee8b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:14.987066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f09cbd-095e-4fd5-8b5f-4ad82324ee8b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:14.987332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f09cbd-095e-4fd5-8b5f-4ad82324ee8b None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:14.987986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f09cbd-095e-4fd5-8b5f-4ad82324ee8b None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:14.988283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f09cbd-095e-4fd5-8b5f-4ad82324ee8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:14.988780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d95d8b-2e33-48e6-a4b1-2b6b780aa4db None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:14.989654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f09cbd-095e-4fd5-8b5f-4ad82324ee8b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:14.991782 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:14.998845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 271/568] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:14.999668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f09cbd-095e-4fd5-8b5f-4ad82324ee8b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.024001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33180b09-34da-450a-8d63-1b2fa46c48b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.024635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33180b09-34da-450a-8d63-1b2fa46c48b7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.024913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33180b09-34da-450a-8d63-1b2fa46c48b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.025187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33180b09-34da-450a-8d63-1b2fa46c48b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.032082 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33180b09-34da-450a-8d63-1b2fa46c48b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.033614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33180b09-34da-450a-8d63-1b2fa46c48b7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-33180b09-34da-450a-8d63-1b2fa46c48b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:15.044662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:15.045644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:15.047028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:15.047028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:15.047028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:15.047028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:15.047028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:15.047028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:15.047028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:15.047028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:15.047028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 298/569] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:15.047563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46c514-d5ee-4e27-b38e-3490aaa051d0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.047563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46c514-d5ee-4e27-b38e-3490aaa051d0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.047781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f46c514-d5ee-4e27-b38e-3490aaa051d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.065393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4f46c514-d5ee-4e27-b38e-3490aaa051d0 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:15.066626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4f46c514-d5ee-4e27-b38e-3490aaa051d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:15.068611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:15.105658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f09cbd-095e-4fd5-8b5f-4ad82324ee8b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.135828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 299/570] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:15.137160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46c514-d5ee-4e27-b38e-3490aaa051d0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.139045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2c5f8-a3b3-4736-b80d-f580efe8596c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.139706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2c5f8-a3b3-4736-b80d-f580efe8596c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.139979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2c5f8-a3b3-4736-b80d-f580efe8596c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.140245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb2c5f8-a3b3-4736-b80d-f580efe8596c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.151533 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2c5f8-a3b3-4736-b80d-f580efe8596c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.153078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb2c5f8-a3b3-4736-b80d-f580efe8596c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9eb2c5f8-a3b3-4736-b80d-f580efe8596c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:15.167404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:15.168681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:15.169722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:15.169722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:15.169722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:15.169722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 272/571] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:15.169722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2c5f8-a3b3-4736-b80d-f580efe8596c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.233022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:15.267014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f46c514-d5ee-4e27-b38e-3490aaa051d0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.287290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb2c5f8-a3b3-4736-b80d-f580efe8596c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.303909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296bf2b-7308-4d92-be89-b0c428c6b4b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.304532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296bf2b-7308-4d92-be89-b0c428c6b4b9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.304793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296bf2b-7308-4d92-be89-b0c428c6b4b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.305068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296bf2b-7308-4d92-be89-b0c428c6b4b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.305735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296bf2b-7308-4d92-be89-b0c428c6b4b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:15.306037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296bf2b-7308-4d92-be89-b0c428c6b4b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.307455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a296bf2b-7308-4d92-be89-b0c428c6b4b9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.312937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:15.321614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 300/572] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:15.322600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296bf2b-7308-4d92-be89-b0c428c6b4b9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.326956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83629e-88de-40a1-9279-1095c8fbcb17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.327606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83629e-88de-40a1-9279-1095c8fbcb17 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.327892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83629e-88de-40a1-9279-1095c8fbcb17 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.328167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e83629e-88de-40a1-9279-1095c8fbcb17 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.339643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83629e-88de-40a1-9279-1095c8fbcb17 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.341177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e83629e-88de-40a1-9279-1095c8fbcb17 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0e83629e-88de-40a1-9279-1095c8fbcb17 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:15.355894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:15.356820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:15.362573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:15.362573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:15.362573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:15.362573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 273/573] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:15.362573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83629e-88de-40a1-9279-1095c8fbcb17 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.423291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296bf2b-7308-4d92-be89-b0c428c6b4b9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.456393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18347d3a-2d79-42e1-8b4c-24630da1299e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.457080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347d3a-2d79-42e1-8b4c-24630da1299e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.457262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347d3a-2d79-42e1-8b4c-24630da1299e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.457534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18347d3a-2d79-42e1-8b4c-24630da1299e None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.458262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18347d3a-2d79-42e1-8b4c-24630da1299e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:15.458572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18347d3a-2d79-42e1-8b4c-24630da1299e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.459216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e83629e-88de-40a1-9279-1095c8fbcb17 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.460012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18347d3a-2d79-42e1-8b4c-24630da1299e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.460750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:15.472552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 301/574] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:15.473360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18347d3a-2d79-42e1-8b4c-24630da1299e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.499685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc39f26-f94d-47dc-aed8-73ec94abc860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.500251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc39f26-f94d-47dc-aed8-73ec94abc860 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.500529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc39f26-f94d-47dc-aed8-73ec94abc860 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.500799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc39f26-f94d-47dc-aed8-73ec94abc860 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.501459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc39f26-f94d-47dc-aed8-73ec94abc860 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:15.501730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc39f26-f94d-47dc-aed8-73ec94abc860 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.503137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc39f26-f94d-47dc-aed8-73ec94abc860 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.514933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 274/575] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:15.515727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc39f26-f94d-47dc-aed8-73ec94abc860 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.534232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:15.577140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18347d3a-2d79-42e1-8b4c-24630da1299e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.615117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cb552-dc4a-4219-9a0a-2cd4287cc512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.615763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cb552-dc4a-4219-9a0a-2cd4287cc512 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.616044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cb552-dc4a-4219-9a0a-2cd4287cc512 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.616319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161cb552-dc4a-4219-9a0a-2cd4287cc512 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.616780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc39f26-f94d-47dc-aed8-73ec94abc860 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.626589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cb552-dc4a-4219-9a0a-2cd4287cc512 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.628056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161cb552-dc4a-4219-9a0a-2cd4287cc512 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-161cb552-dc4a-4219-9a0a-2cd4287cc512 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:15.642766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:15.643867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:15.645111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:15.645111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:15.645111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:15.645111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 302/576] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:15.645111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cb552-dc4a-4219-9a0a-2cd4287cc512 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.655209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4651d5-cb99-43e5-8b0a-1edf412a0a0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.655834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4651d5-cb99-43e5-8b0a-1edf412a0a0d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.656118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4651d5-cb99-43e5-8b0a-1edf412a0a0d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.656394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4651d5-cb99-43e5-8b0a-1edf412a0a0d None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.657049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4651d5-cb99-43e5-8b0a-1edf412a0a0d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:15.657354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4651d5-cb99-43e5-8b0a-1edf412a0a0d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.658791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4651d5-cb99-43e5-8b0a-1edf412a0a0d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.670111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 275/577] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:15.671205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4651d5-cb99-43e5-8b0a-1edf412a0a0d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.696600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:15.744342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161cb552-dc4a-4219-9a0a-2cd4287cc512 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.756987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:15.771330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4651d5-cb99-43e5-8b0a-1edf412a0a0d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.778499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e302bf1-1fa5-4bdc-a752-cd2f07367a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.779137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e302bf1-1fa5-4bdc-a752-cd2f07367a2e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.779419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e302bf1-1fa5-4bdc-a752-cd2f07367a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.779693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e302bf1-1fa5-4bdc-a752-cd2f07367a2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.789921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e302bf1-1fa5-4bdc-a752-cd2f07367a2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0e302bf1-1fa5-4bdc-a752-cd2f07367a2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:15.795284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:15.796232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:15.797071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 303/578] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:15.797959 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e302bf1-1fa5-4bdc-a752-cd2f07367a2e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.809090 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923de4-610d-4a36-b2eb-2c94e0455c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.809824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923de4-610d-4a36-b2eb-2c94e0455c4b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.810113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923de4-610d-4a36-b2eb-2c94e0455c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.810385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7923de4-610d-4a36-b2eb-2c94e0455c4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.811105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7923de4-610d-4a36-b2eb-2c94e0455c4b None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=4815ef133a3a4604bc467c222d450fbf) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:15.811404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7923de4-610d-4a36-b2eb-2c94e0455c4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.812836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7923de4-610d-4a36-b2eb-2c94e0455c4b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.818280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 276/579] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/role_assignments?user.id=4815ef133a3a4604bc467c222d450fbf&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:15.819045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923de4-610d-4a36-b2eb-2c94e0455c4b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:15.921477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e302bf1-1fa5-4bdc-a752-cd2f07367a2e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.922648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:15.939915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7923de4-610d-4a36-b2eb-2c94e0455c4b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:15.958960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eceab-24d8-4391-bd9b-04eb7bd43e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.959603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eceab-24d8-4391-bd9b-04eb7bd43e77 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.959877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eceab-24d8-4391-bd9b-04eb7bd43e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.960145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00eceab-24d8-4391-bd9b-04eb7bd43e77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.967615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00eceab-24d8-4391-bd9b-04eb7bd43e77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.969115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00eceab-24d8-4391-bd9b-04eb7bd43e77 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.978107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec82ff8-5a60-4ba7-bddb-c02b65e37c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:15.978745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec82ff8-5a60-4ba7-bddb-c02b65e37c83 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:15.979028 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec82ff8-5a60-4ba7-bddb-c02b65e37c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:15.979299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec82ff8-5a60-4ba7-bddb-c02b65e37c83 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:15.979951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec82ff8-5a60-4ba7-bddb-c02b65e37c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:15.980259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec82ff8-5a60-4ba7-bddb-c02b65e37c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a00eceab-24d8-4391-bd9b-04eb7bd43e77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:15.981269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:15.982166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:15.987985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:15.987985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:15.987985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:15.987985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:15.987985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:15.987985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:15.987985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:15.987985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:15.988516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec82ff8-5a60-4ba7-bddb-c02b65e37c83 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:15.988601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 304/580] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:15.988601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eceab-24d8-4391-bd9b-04eb7bd43e77 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.002144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 277/581] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:16.003868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec82ff8-5a60-4ba7-bddb-c02b65e37c83 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.024724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:16.101241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00eceab-24d8-4391-bd9b-04eb7bd43e77 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.108613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec82ff8-5a60-4ba7-bddb-c02b65e37c83 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.139806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099040c-592d-4cd4-b592-824734b3966b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.140549 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099040c-592d-4cd4-b592-824734b3966b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.140830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099040c-592d-4cd4-b592-824734b3966b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.141098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2099040c-592d-4cd4-b592-824734b3966b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.144359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7edc1-7d9a-4e33-bda6-92a91f7d00a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.144967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7edc1-7d9a-4e33-bda6-92a91f7d00a7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.145243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7edc1-7d9a-4e33-bda6-92a91f7d00a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.145509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf7edc1-7d9a-4e33-bda6-92a91f7d00a7 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.146195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7edc1-7d9a-4e33-bda6-92a91f7d00a7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:16.146494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7edc1-7d9a-4e33-bda6-92a91f7d00a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.147861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf7edc1-7d9a-4e33-bda6-92a91f7d00a7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.149043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099040c-592d-4cd4-b592-824734b3966b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.150583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2099040c-592d-4cd4-b592-824734b3966b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.162405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 278/582] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2099040c-592d-4cd4-b592-824734b3966b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:16.162596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:16.163474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 06:48:16.164432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:16.165017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7edc1-7d9a-4e33-bda6-92a91f7d00a7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.166398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 305/583] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:16.168456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099040c-592d-4cd4-b592-824734b3966b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.175036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:16.266923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:16.268615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf7edc1-7d9a-4e33-bda6-92a91f7d00a7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.276708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2099040c-592d-4cd4-b592-824734b3966b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.305395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63616bd7-2128-4464-939b-147801cc4961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.306395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63616bd7-2128-4464-939b-147801cc4961 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.306694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63616bd7-2128-4464-939b-147801cc4961 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.306949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63616bd7-2128-4464-939b-147801cc4961 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.307646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63616bd7-2128-4464-939b-147801cc4961 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:16.308303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63616bd7-2128-4464-939b-147801cc4961 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.309836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63616bd7-2128-4464-939b-147801cc4961 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.312927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f3d8b-6731-4bcd-bd63-eeed35d2421d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.313745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3d8b-6731-4bcd-bd63-eeed35d2421d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.314182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3d8b-6731-4bcd-bd63-eeed35d2421d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.314614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415f3d8b-6731-4bcd-bd63-eeed35d2421d None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.322625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f3d8b-6731-4bcd-bd63-eeed35d2421d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.324272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415f3d8b-6731-4bcd-bd63-eeed35d2421d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.328138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 279/584] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:16.328972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63616bd7-2128-4464-939b-147801cc4961 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-415f3d8b-6731-4bcd-bd63-eeed35d2421d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:16.338496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:16.339639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 06:48:16.340860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:16.342666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 306/585] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:16.343653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f3d8b-6731-4bcd-bd63-eeed35d2421d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.407579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:16.445554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63616bd7-2128-4464-939b-147801cc4961 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.448424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415f3d8b-6731-4bcd-bd63-eeed35d2421d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.482452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc4ddb-ccf5-4fd6-a3a1-c3c7fe99a98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.482926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bf5b8-4fe0-4fa3-afe7-1ec6580c7b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.483391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc4ddb-ccf5-4fd6-a3a1-c3c7fe99a98f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.483391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc4ddb-ccf5-4fd6-a3a1-c3c7fe99a98f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.483556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bf5b8-4fe0-4fa3-afe7-1ec6580c7b48 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.483630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc4ddb-ccf5-4fd6-a3a1-c3c7fe99a98f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.483837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bf5b8-4fe0-4fa3-afe7-1ec6580c7b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.484114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402bf5b8-4fe0-4fa3-afe7-1ec6580c7b48 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.484775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bf5b8-4fe0-4fa3-afe7-1ec6580c7b48 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:16.485080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bf5b8-4fe0-4fa3-afe7-1ec6580c7b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.486590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402bf5b8-4fe0-4fa3-afe7-1ec6580c7b48 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.494101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc4ddb-ccf5-4fd6-a3a1-c3c7fe99a98f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.495572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc4ddb-ccf5-4fd6-a3a1-c3c7fe99a98f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.504828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 307/586] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:16.505667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bf5b8-4fe0-4fa3-afe7-1ec6580c7b48 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-46cc4ddb-ccf5-4fd6-a3a1-c3c7fe99a98f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:16.514142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:16.515124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:16.516010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:16.516010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:16.516010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:16.516188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 280/587] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:16.517039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc4ddb-ccf5-4fd6-a3a1-c3c7fe99a98f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.520278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:16.617679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402bf5b8-4fe0-4fa3-afe7-1ec6580c7b48 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.627133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc4ddb-ccf5-4fd6-a3a1-c3c7fe99a98f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.646722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:16.656389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef25166-81db-4a75-92b4-c0399b8d90d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.657131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef25166-81db-4a75-92b4-c0399b8d90d0 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.657407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef25166-81db-4a75-92b4-c0399b8d90d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.657674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef25166-81db-4a75-92b4-c0399b8d90d0 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/87f3a9ffb88f4faa85edc3dabce0c8f5/roles/df32bbf3791f4028b2dabf497d5e938d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.661562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901d746-b1cc-47a6-9180-623c95914f2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.662203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d746-b1cc-47a6-9180-623c95914f2a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.662481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d746-b1cc-47a6-9180-623c95914f2a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.662749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1901d746-b1cc-47a6-9180-623c95914f2a None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.663472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901d746-b1cc-47a6-9180-623c95914f2a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:16.663775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901d746-b1cc-47a6-9180-623c95914f2a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.665212 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1901d746-b1cc-47a6-9180-623c95914f2a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.672192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef25166-81db-4a75-92b4-c0399b8d90d0 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=87f3a9ffb88f4faa85edc3dabce0c8f5, role_id=df32bbf3791f4028b2dabf497d5e938d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.673651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef25166-81db-4a75-92b4-c0399b8d90d0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.689344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 281/588] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:16.689344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901d746-b1cc-47a6-9180-623c95914f2a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.695045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 308/589] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:48:16 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/87f3a9ffb88f4faa85edc3dabce0c8f5/roles/df32bbf3791f4028b2dabf497d5e938d => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:48:16.703171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef25166-81db-4a75-92b4-c0399b8d90d0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.775333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:16.836294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1901d746-b1cc-47a6-9180-623c95914f2a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.837805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef25166-81db-4a75-92b4-c0399b8d90d0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:16.877742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5291a42-ee6f-4685-add7-7ae51d097b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.878860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5291a42-ee6f-4685-add7-7ae51d097b4a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.879336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5291a42-ee6f-4685-add7-7ae51d097b4a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.879803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5291a42-ee6f-4685-add7-7ae51d097b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.892447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41891242-78a3-4d5e-8cd7-06d5a1740a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:16.892735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41891242-78a3-4d5e-8cd7-06d5a1740a88 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:16.892831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41891242-78a3-4d5e-8cd7-06d5a1740a88 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:16.893330 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41891242-78a3-4d5e-8cd7-06d5a1740a88 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:16.893910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5291a42-ee6f-4685-add7-7ae51d097b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.894021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41891242-78a3-4d5e-8cd7-06d5a1740a88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:16.894321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41891242-78a3-4d5e-8cd7-06d5a1740a88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:16.895427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5291a42-ee6f-4685-add7-7ae51d097b4a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.895731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41891242-78a3-4d5e-8cd7-06d5a1740a88 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:16.910053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 309/590] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:16.911434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41891242-78a3-4d5e-8cd7-06d5a1740a88 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f5291a42-ee6f-4685-add7-7ae51d097b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:16.913668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:16.914844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:16.918111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:16.918111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:16.918111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:16.918111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 282/591] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:16.918111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5291a42-ee6f-4685-add7-7ae51d097b4a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:16.953540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:17.024861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41891242-78a3-4d5e-8cd7-06d5a1740a88 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.027026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5291a42-ee6f-4685-add7-7ae51d097b4a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.047344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:17.070310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7df83c-a161-410c-8f88-6b0613cace95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.071480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7df83c-a161-410c-8f88-6b0613cace95 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.072077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7df83c-a161-410c-8f88-6b0613cace95 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.072551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7df83c-a161-410c-8f88-6b0613cace95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.075607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11817642-6876-4400-8a99-4efdbc72fd36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.076506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11817642-6876-4400-8a99-4efdbc72fd36 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.076506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11817642-6876-4400-8a99-4efdbc72fd36 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.076787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11817642-6876-4400-8a99-4efdbc72fd36 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.084301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11817642-6876-4400-8a99-4efdbc72fd36 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:17.085752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11817642-6876-4400-8a99-4efdbc72fd36 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:17.087345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7df83c-a161-410c-8f88-6b0613cace95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:17.089661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7df83c-a161-410c-8f88-6b0613cace95 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-11817642-6876-4400-8a99-4efdbc72fd36 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:17.098955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:17.099925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:17.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 283/592] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/users/neutron => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:17.102716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11817642-6876-4400-8a99-4efdbc72fd36 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4d7df83c-a161-410c-8f88-6b0613cace95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:17.110964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:17.112152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:17.113245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:17.113245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:17.113245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:17.115483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 310/593] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:17.116639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7df83c-a161-410c-8f88-6b0613cace95 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:17.219362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:17.232106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11817642-6876-4400-8a99-4efdbc72fd36 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.246004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7df83c-a161-410c-8f88-6b0613cace95 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.278532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89963f14-ded6-40c7-9ada-5ac03e463ac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.279576 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89963f14-ded6-40c7-9ada-5ac03e463ac3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.280001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89963f14-ded6-40c7-9ada-5ac03e463ac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.280516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89963f14-ded6-40c7-9ada-5ac03e463ac3 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.281508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89963f14-ded6-40c7-9ada-5ac03e463ac3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:17.282024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89963f14-ded6-40c7-9ada-5ac03e463ac3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:17.282461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a7115-660e-4a87-8af8-329f65517dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.283165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a7115-660e-4a87-8af8-329f65517dad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.283534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a7115-660e-4a87-8af8-329f65517dad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.283897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23a7115-660e-4a87-8af8-329f65517dad None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.284242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89963f14-ded6-40c7-9ada-5ac03e463ac3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:17.284802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a7115-660e-4a87-8af8-329f65517dad None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:17.285197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a7115-660e-4a87-8af8-329f65517dad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:17.286857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23a7115-660e-4a87-8af8-329f65517dad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:17.298874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 284/594] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:17.300059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89963f14-ded6-40c7-9ada-5ac03e463ac3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:17.309001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 311/595] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Sep 13 06:48:17 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:17.310531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a7115-660e-4a87-8af8-329f65517dad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:17.348412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:17.411643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89963f14-ded6-40c7-9ada-5ac03e463ac3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.429965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23a7115-660e-4a87-8af8-329f65517dad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.456182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7914a8-d3c1-4476-82f3-c572781edc25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.456558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7914a8-d3c1-4476-82f3-c572781edc25 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.456935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7914a8-d3c1-4476-82f3-c572781edc25 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.457329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7914a8-d3c1-4476-82f3-c572781edc25 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.458072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7914a8-d3c1-4476-82f3-c572781edc25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:17.458530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7914a8-d3c1-4476-82f3-c572781edc25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:17.460105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7914a8-d3c1-4476-82f3-c572781edc25 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:17.473024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 285/596] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:17.474099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7914a8-d3c1-4476-82f3-c572781edc25 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:17.475146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd421152-90b9-4440-ac7f-376394e6a66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.475954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd421152-90b9-4440-ac7f-376394e6a66e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.476393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd421152-90b9-4440-ac7f-376394e6a66e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.476891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd421152-90b9-4440-ac7f-376394e6a66e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.486448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd421152-90b9-4440-ac7f-376394e6a66e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:17.488829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:17.489143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd421152-90b9-4440-ac7f-376394e6a66e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-dd421152-90b9-4440-ac7f-376394e6a66e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:17.505649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:17.506817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:17.507912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:17.507912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:17.507912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:17.507912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:17.507912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:17.507912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:17.507912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:17.507912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:17.510436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 312/597] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:17.511422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd421152-90b9-4440-ac7f-376394e6a66e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:17.594718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7914a8-d3c1-4476-82f3-c572781edc25 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.633399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:17.658508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd421152-90b9-4440-ac7f-376394e6a66e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.669526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104ba6c-6b7a-4b08-9ba9-d65e22da4c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.670369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104ba6c-6b7a-4b08-9ba9-d65e22da4c1b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.670746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104ba6c-6b7a-4b08-9ba9-d65e22da4c1b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.671154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2104ba6c-6b7a-4b08-9ba9-d65e22da4c1b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.682732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104ba6c-6b7a-4b08-9ba9-d65e22da4c1b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:17.684808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2104ba6c-6b7a-4b08-9ba9-d65e22da4c1b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2104ba6c-6b7a-4b08-9ba9-d65e22da4c1b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:17.702150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:17.703317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:17.704194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:17.704194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:17.704194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:17.704497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 286/598] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:17.705515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104ba6c-6b7a-4b08-9ba9-d65e22da4c1b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:17.721382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f93d64-7cb6-47ac-b4c9-75a678880133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.722829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f93d64-7cb6-47ac-b4c9-75a678880133 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.723156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f93d64-7cb6-47ac-b4c9-75a678880133 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.723371 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f93d64-7cb6-47ac-b4c9-75a678880133 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.739080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f93d64-7cb6-47ac-b4c9-75a678880133 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:17.741138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f93d64-7cb6-47ac-b4c9-75a678880133 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a7f93d64-7cb6-47ac-b4c9-75a678880133 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:17.759943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:17.760862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:17.762482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:17.762482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:17.762482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:17.762482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:17.762482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:17.762482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:17.762482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:17.762482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:17.762482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 313/599] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:17.768614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f93d64-7cb6-47ac-b4c9-75a678880133 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:17.816372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:17.867172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2104ba6c-6b7a-4b08-9ba9-d65e22da4c1b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.928233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f93d64-7cb6-47ac-b4c9-75a678880133 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:17.933385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd91e05-baab-4c20-982a-23bed8cdc2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.934670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd91e05-baab-4c20-982a-23bed8cdc2b0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.936894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd91e05-baab-4c20-982a-23bed8cdc2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.937424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd91e05-baab-4c20-982a-23bed8cdc2b0 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.939408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd91e05-baab-4c20-982a-23bed8cdc2b0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:17.939966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd91e05-baab-4c20-982a-23bed8cdc2b0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:17.945471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd91e05-baab-4c20-982a-23bed8cdc2b0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:17.970369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 287/600] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:17.971621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd91e05-baab-4c20-982a-23bed8cdc2b0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:17.996998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8e1f8-86a7-4567-ba76-2aa0d5614c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:17.997709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8e1f8-86a7-4567-ba76-2aa0d5614c40 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:17.997986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8e1f8-86a7-4567-ba76-2aa0d5614c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:17.998288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8e1f8-86a7-4567-ba76-2aa0d5614c40 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:17.998940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8e1f8-86a7-4567-ba76-2aa0d5614c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:17.999281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8e1f8-86a7-4567-ba76-2aa0d5614c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:18.000915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8e1f8-86a7-4567-ba76-2aa0d5614c40 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:18.020031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 314/601] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:18.020869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8e1f8-86a7-4567-ba76-2aa0d5614c40 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:18.044716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:18.107435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd91e05-baab-4c20-982a-23bed8cdc2b0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:18.133944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:18.147756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9d1e4f-7a74-48f8-8147-1421f948a4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:18.148372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d1e4f-7a74-48f8-8147-1421f948a4f0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:18.148647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d1e4f-7a74-48f8-8147-1421f948a4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:18.148921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9d1e4f-7a74-48f8-8147-1421f948a4f0 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:18.149606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d1e4f-7a74-48f8-8147-1421f948a4f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:18.149912 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d1e4f-7a74-48f8-8147-1421f948a4f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:18.151559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9d1e4f-7a74-48f8-8147-1421f948a4f0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:18.164895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 288/602] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:18.165031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8e1f8-86a7-4567-ba76-2aa0d5614c40 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:18.166588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c29ef-76af-47dc-ae6d-64437451a99d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:18.167077 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c29ef-76af-47dc-ae6d-64437451a99d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:18.167642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3c29ef-76af-47dc-ae6d-64437451a99d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:18.173611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 289/603] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:18 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:18.174429 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c29ef-76af-47dc-ae6d-64437451a99d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:18.206781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c483a6f-b3f9-4dc4-ac1e-868e991bf174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:18.207454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483a6f-b3f9-4dc4-ac1e-868e991bf174 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:18.207734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483a6f-b3f9-4dc4-ac1e-868e991bf174 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:18.208068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c483a6f-b3f9-4dc4-ac1e-868e991bf174 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/88520ce6493a44f39b19c67592f03632/roles/df32bbf3791f4028b2dabf497d5e938d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:18.222532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c483a6f-b3f9-4dc4-ac1e-868e991bf174 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=88520ce6493a44f39b19c67592f03632, role_id=df32bbf3791f4028b2dabf497d5e938d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:18.224119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c483a6f-b3f9-4dc4-ac1e-868e991bf174 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:18.246119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 315/604] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:48:18 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/88520ce6493a44f39b19c67592f03632/roles/df32bbf3791f4028b2dabf497d5e938d => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:48:18.246119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c483a6f-b3f9-4dc4-ac1e-868e991bf174 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:18.280978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3c29ef-76af-47dc-ae6d-64437451a99d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:18.318473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a8e19-f457-49c4-84f3-bd8aabf699fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:18.319126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a8e19-f457-49c4-84f3-bd8aabf699fb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:18.319405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a8e19-f457-49c4-84f3-bd8aabf699fb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:18.319674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a8e19-f457-49c4-84f3-bd8aabf699fb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:18.330626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a8e19-f457-49c4-84f3-bd8aabf699fb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:18.332163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a8e19-f457-49c4-84f3-bd8aabf699fb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:18.336977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-421a8e19-f457-49c4-84f3-bd8aabf699fb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:18.346284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:18.347269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:18.348186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:18.348186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:18.348186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:18.348186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:18.348186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:18.348186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:18.348186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:18.348186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:18.348186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 290/605] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:18.348669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d3c1-d8b0-4dea-808b-576dd27f9273 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:18.354374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d3c1-d8b0-4dea-808b-576dd27f9273 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:18.354374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc9d3c1-d8b0-4dea-808b-576dd27f9273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:18.379199 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2bc9d3c1-d8b0-4dea-808b-576dd27f9273 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:18.380448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2bc9d3c1-d8b0-4dea-808b-576dd27f9273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:18.404845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c483a6f-b3f9-4dc4-ac1e-868e991bf174 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:18.432369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:18.449562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671a722-439e-458c-9aeb-cee1e9da6c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:18.450457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671a722-439e-458c-9aeb-cee1e9da6c3e None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:18.450457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671a722-439e-458c-9aeb-cee1e9da6c3e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:18.450759 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8671a722-439e-458c-9aeb-cee1e9da6c3e None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/3505c8965c4a42e493a2f5916cab56dc/roles/df32bbf3791f4028b2dabf497d5e938d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:18.471771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671a722-439e-458c-9aeb-cee1e9da6c3e None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=3505c8965c4a42e493a2f5916cab56dc, role_id=df32bbf3791f4028b2dabf497d5e938d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:18.473260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8671a722-439e-458c-9aeb-cee1e9da6c3e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:18.479660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 291/606] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:18.480678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9d3c1-d8b0-4dea-808b-576dd27f9273 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:18.498184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 316/607] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:48:18 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/3505c8965c4a42e493a2f5916cab56dc/roles/df32bbf3791f4028b2dabf497d5e938d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:48:18.624719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc9d3c1-d8b0-4dea-808b-576dd27f9273 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:18.682470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b844d70-eb46-4802-a786-6bf5fd5d6cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:18.683505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b844d70-eb46-4802-a786-6bf5fd5d6cd8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:18.683808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b844d70-eb46-4802-a786-6bf5fd5d6cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:18.684114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b844d70-eb46-4802-a786-6bf5fd5d6cd8 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:18.684810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b844d70-eb46-4802-a786-6bf5fd5d6cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:18.685111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b844d70-eb46-4802-a786-6bf5fd5d6cd8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:18.687138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b844d70-eb46-4802-a786-6bf5fd5d6cd8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:18.701436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 292/608] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:18.706980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671a722-439e-458c-9aeb-cee1e9da6c3e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:18.846269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8671a722-439e-458c-9aeb-cee1e9da6c3e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:18.858034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:18.889527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa26c9fe-ae35-43bb-a18b-9170c92e81a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:18.890611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26c9fe-ae35-43bb-a18b-9170c92e81a3 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:18.891101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26c9fe-ae35-43bb-a18b-9170c92e81a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:18.891566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa26c9fe-ae35-43bb-a18b-9170c92e81a3 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/f94aa8af227a42a08d8a0e4b7f02b85a/roles/df32bbf3791f4028b2dabf497d5e938d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:18.911312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26c9fe-ae35-43bb-a18b-9170c92e81a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=f94aa8af227a42a08d8a0e4b7f02b85a, role_id=df32bbf3791f4028b2dabf497d5e938d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:18.913109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa26c9fe-ae35-43bb-a18b-9170c92e81a3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:18.923591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b844d70-eb46-4802-a786-6bf5fd5d6cd8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:18.948801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 317/609] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:48:18 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/f94aa8af227a42a08d8a0e4b7f02b85a/roles/df32bbf3791f4028b2dabf497d5e938d => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:48:18.974501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:19.055221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b844d70-eb46-4802-a786-6bf5fd5d6cd8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:19.092895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2864386-8232-4e9a-9be9-0b190d50a121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:19.093558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2864386-8232-4e9a-9be9-0b190d50a121 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:19.093838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2864386-8232-4e9a-9be9-0b190d50a121 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:19.094143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2864386-8232-4e9a-9be9-0b190d50a121 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:19.103745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fef6f-79f4-4285-b1da-bfa53461de8c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:19.104078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fef6f-79f4-4285-b1da-bfa53461de8c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:19.104296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073fef6f-79f4-4285-b1da-bfa53461de8c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:19.106166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 318/610] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:19.106318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2864386-8232-4e9a-9be9-0b190d50a121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a2864386-8232-4e9a-9be9-0b190d50a121 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:19.111966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:19.112973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:19.113804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 293/611] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:19.115132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ea5d8-a0ff-4997-8500-b8653bf29553 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:19.115292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ea5d8-a0ff-4997-8500-b8653bf29553 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:19.115535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71ea5d8-a0ff-4997-8500-b8653bf29553 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:19.119274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2864386-8232-4e9a-9be9-0b190d50a121 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:19.163618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a71ea5d8-a0ff-4997-8500-b8653bf29553 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:19.164758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a71ea5d8-a0ff-4997-8500-b8653bf29553 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:19.273595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:19.283473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2864386-8232-4e9a-9be9-0b190d50a121 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:19.294812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 319/612] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:19.322367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d2b02-ae07-4582-ae9f-e65de89d0449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:19.322986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d2b02-ae07-4582-ae9f-e65de89d0449 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:19.323263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d2b02-ae07-4582-ae9f-e65de89d0449 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:19.323588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4d2b02-ae07-4582-ae9f-e65de89d0449 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:19.324255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d2b02-ae07-4582-ae9f-e65de89d0449 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:19.324585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d2b02-ae07-4582-ae9f-e65de89d0449 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:19.326276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4d2b02-ae07-4582-ae9f-e65de89d0449 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:19.335040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 294/613] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:19.343324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ea5d8-a0ff-4997-8500-b8653bf29553 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:19.470935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:19.482029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71ea5d8-a0ff-4997-8500-b8653bf29553 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:19.527229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee177ad-8cb3-45a3-b69e-c70c2570380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:19.528052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee177ad-8cb3-45a3-b69e-c70c2570380f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:19.528433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee177ad-8cb3-45a3-b69e-c70c2570380f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:19.528722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee177ad-8cb3-45a3-b69e-c70c2570380f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:19.542796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee177ad-8cb3-45a3-b69e-c70c2570380f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:19.544407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee177ad-8cb3-45a3-b69e-c70c2570380f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-5ee177ad-8cb3-45a3-b69e-c70c2570380f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:19.563338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:19.564301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:19.565201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:19.565201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:19.565201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:19.565201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 320/614] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:19.570206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d2b02-ae07-4582-ae9f-e65de89d0449 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:19.596738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee177ad-8cb3-45a3-b69e-c70c2570380f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:19.711637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee177ad-8cb3-45a3-b69e-c70c2570380f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:19.751612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c8e3b-2115-437b-af44-3e8f462c86f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:19.752597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c8e3b-2115-437b-af44-3e8f462c86f1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:19.753084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c8e3b-2115-437b-af44-3e8f462c86f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:19.753572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5c8e3b-2115-437b-af44-3e8f462c86f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:19.765452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5c8e3b-2115-437b-af44-3e8f462c86f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8e5c8e3b-2115-437b-af44-3e8f462c86f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:19.772101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:19.773022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:19.773874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:19.781331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 321/615] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:19.782751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c8e3b-2115-437b-af44-3e8f462c86f1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:19.810451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:19.832104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4d2b02-ae07-4582-ae9f-e65de89d0449 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:19.893840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5c8e3b-2115-437b-af44-3e8f462c86f1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:19.932152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f68a16-646e-4191-9c7a-5cb9c0a6c518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:19.933534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68a16-646e-4191-9c7a-5cb9c0a6c518 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:19.934273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68a16-646e-4191-9c7a-5cb9c0a6c518 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:19.934829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f68a16-646e-4191-9c7a-5cb9c0a6c518 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:19.935776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f68a16-646e-4191-9c7a-5cb9c0a6c518 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:19.936291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f68a16-646e-4191-9c7a-5cb9c0a6c518 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:19.937983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f68a16-646e-4191-9c7a-5cb9c0a6c518 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:19.950824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea83cf-98a2-4b76-82ff-6e31a7d8cd91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:19.951202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 322/616] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:19.952145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea83cf-98a2-4b76-82ff-6e31a7d8cd91 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:19.952145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea83cf-98a2-4b76-82ff-6e31a7d8cd91 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:19.952145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ea83cf-98a2-4b76-82ff-6e31a7d8cd91 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:19.952881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea83cf-98a2-4b76-82ff-6e31a7d8cd91 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:19.953050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea83cf-98a2-4b76-82ff-6e31a7d8cd91 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:19.954157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f68a16-646e-4191-9c7a-5cb9c0a6c518 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:19.954415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ea83cf-98a2-4b76-82ff-6e31a7d8cd91 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:19.988337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 295/617] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:19.999133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea83cf-98a2-4b76-82ff-6e31a7d8cd91 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:20.043697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:20.048373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:20.072476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f68a16-646e-4191-9c7a-5cb9c0a6c518 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:20.140667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01b09e-2fde-409d-b7b7-a2f3444aaa46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:20.141669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01b09e-2fde-409d-b7b7-a2f3444aaa46 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:20.142190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01b09e-2fde-409d-b7b7-a2f3444aaa46 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:20.142678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c01b09e-2fde-409d-b7b7-a2f3444aaa46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:20.158658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01b09e-2fde-409d-b7b7-a2f3444aaa46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:20.160397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c01b09e-2fde-409d-b7b7-a2f3444aaa46 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6c01b09e-2fde-409d-b7b7-a2f3444aaa46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:20.176396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:20.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:20.178313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:20.178313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:20.178313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:20.184659 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 323/618] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:20.187530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01b09e-2fde-409d-b7b7-a2f3444aaa46 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:20.242711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ea83cf-98a2-4b76-82ff-6e31a7d8cd91 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:20.280742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b111d21-9be7-4133-9535-daf2114769a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:20.281748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b111d21-9be7-4133-9535-daf2114769a7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:20.282391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b111d21-9be7-4133-9535-daf2114769a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:20.282860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b111d21-9be7-4133-9535-daf2114769a7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:20.291019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b111d21-9be7-4133-9535-daf2114769a7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:20.292729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b111d21-9be7-4133-9535-daf2114769a7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0b111d21-9be7-4133-9535-daf2114769a7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:20.305950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:20.307066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:20.308385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:20.311034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 296/619] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 312 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:20.319129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b111d21-9be7-4133-9535-daf2114769a7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:20.384340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c01b09e-2fde-409d-b7b7-a2f3444aaa46 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:20.420760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:20.428540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:20.448882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b111d21-9be7-4133-9535-daf2114769a7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:20.482830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cb1d0-ce29-4135-94c6-419dbb100e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:20.483472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cb1d0-ce29-4135-94c6-419dbb100e36 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:20.483744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cb1d0-ce29-4135-94c6-419dbb100e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:20.484022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927cb1d0-ce29-4135-94c6-419dbb100e36 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:20.484674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cb1d0-ce29-4135-94c6-419dbb100e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:20.484979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cb1d0-ce29-4135-94c6-419dbb100e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:20.486381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927cb1d0-ce29-4135-94c6-419dbb100e36 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:20.487182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56619a-c5c4-4456-a13e-79bdb8ffb65e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:20.488151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56619a-c5c4-4456-a13e-79bdb8ffb65e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:20.488641 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56619a-c5c4-4456-a13e-79bdb8ffb65e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:20.489146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de56619a-c5c4-4456-a13e-79bdb8ffb65e None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:20.490072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56619a-c5c4-4456-a13e-79bdb8ffb65e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:20.490580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56619a-c5c4-4456-a13e-79bdb8ffb65e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:20.492228 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de56619a-c5c4-4456-a13e-79bdb8ffb65e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:20.512294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 324/620] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:20.513562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cb1d0-ce29-4135-94c6-419dbb100e36 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:20.516998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 297/621] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Sep 13 06:48:20 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:20.524104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56619a-c5c4-4456-a13e-79bdb8ffb65e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:20.696635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de56619a-c5c4-4456-a13e-79bdb8ffb65e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:20.706974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927cb1d0-ce29-4135-94c6-419dbb100e36 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:20.743951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac050a4-02fc-4aed-8ad4-d27eae867cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:20.744318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac050a4-02fc-4aed-8ad4-d27eae867cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:20.744599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac050a4-02fc-4aed-8ad4-d27eae867cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:20.744875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac050a4-02fc-4aed-8ad4-d27eae867cd9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:20.756769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac050a4-02fc-4aed-8ad4-d27eae867cd9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:20.756769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac050a4-02fc-4aed-8ad4-d27eae867cd9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:20.756994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:20.764612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cead0-ad4c-4f55-9430-78b00f76aeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:20.765488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cead0-ad4c-4f55-9430-78b00f76aeee None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:20.765970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cead0-ad4c-4f55-9430-78b00f76aeee None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:20.766442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cead0-ad4c-4f55-9430-78b00f76aeee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7ac050a4-02fc-4aed-8ad4-d27eae867cd9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:20.768866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:20.769933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:20.771142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 325/622] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Sep 13 06:48:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:20.774770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac050a4-02fc-4aed-8ad4-d27eae867cd9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:20.790130 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cead0-ad4c-4f55-9430-78b00f76aeee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:20.790130 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1cead0-ad4c-4f55-9430-78b00f76aeee None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-0f1cead0-ad4c-4f55-9430-78b00f76aeee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:20.825162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:20.826237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:20.827396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:20.827396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:20.827396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:20.827396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 298/623] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:20.832645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cead0-ad4c-4f55-9430-78b00f76aeee None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:20.851147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:20.942379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cead0-ad4c-4f55-9430-78b00f76aeee None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:20.976279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac050a4-02fc-4aed-8ad4-d27eae867cd9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:20.980001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b139cb-4c68-4bbb-b238-bda953b42f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:20.980931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b139cb-4c68-4bbb-b238-bda953b42f9c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:20.981360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b139cb-4c68-4bbb-b238-bda953b42f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:20.981779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b139cb-4c68-4bbb-b238-bda953b42f9c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:20.982653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b139cb-4c68-4bbb-b238-bda953b42f9c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:20.983178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b139cb-4c68-4bbb-b238-bda953b42f9c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:20.984884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b139cb-4c68-4bbb-b238-bda953b42f9c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:21.000066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 299/624] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:21.006557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d521b2d-22c6-49d7-ad5a-c814b7beaaca None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.007110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d521b2d-22c6-49d7-ad5a-c814b7beaaca None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.007607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d521b2d-22c6-49d7-ad5a-c814b7beaaca None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.009493 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 300/625] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:21.010546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d521b2d-22c6-49d7-ad5a-c814b7beaaca None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:21.020575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7661ceab-7752-4ea9-877c-033ba6465fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:21.021202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661ceab-7752-4ea9-877c-033ba6465fd4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.021482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661ceab-7752-4ea9-877c-033ba6465fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.021757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7661ceab-7752-4ea9-877c-033ba6465fd4 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.022646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661ceab-7752-4ea9-877c-033ba6465fd4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:21.022945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661ceab-7752-4ea9-877c-033ba6465fd4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:21.024407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7661ceab-7752-4ea9-877c-033ba6465fd4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:21.047987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 326/626] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Sep 13 06:48:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:21.047987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1b380-8b68-4261-a7ff-aeee5ed7e229 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.047987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1b380-8b68-4261-a7ff-aeee5ed7e229 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.047987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c1b380-8b68-4261-a7ff-aeee5ed7e229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.065448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c7c1b380-8b68-4261-a7ff-aeee5ed7e229 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:21.066894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c7c1b380-8b68-4261-a7ff-aeee5ed7e229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:21.069654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:21.128620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d521b2d-22c6-49d7-ad5a-c814b7beaaca None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:21.143686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 327/627] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:21.144563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1b380-8b68-4261-a7ff-aeee5ed7e229 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:21.174395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5924f299-7e73-4cdb-9750-c12883d4995a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:21.175207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924f299-7e73-4cdb-9750-c12883d4995a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.175774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:21.176595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924f299-7e73-4cdb-9750-c12883d4995a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.176595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5924f299-7e73-4cdb-9750-c12883d4995a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.186240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924f299-7e73-4cdb-9750-c12883d4995a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:21.188163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5924f299-7e73-4cdb-9750-c12883d4995a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5924f299-7e73-4cdb-9750-c12883d4995a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:21.201606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:21.202606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:21.203593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:21.203593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:21.203593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:21.203593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:21.203593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:21.203593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:21.203593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:21.203593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:21.207679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 301/628] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:21.209085 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f35ad2-6f76-48af-9aba-ee9c5b0565d1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.209364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f35ad2-6f76-48af-9aba-ee9c5b0565d1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.209632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f35ad2-6f76-48af-9aba-ee9c5b0565d1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.211210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 302/629] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:21.211996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f35ad2-6f76-48af-9aba-ee9c5b0565d1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:21.264817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c1b380-8b68-4261-a7ff-aeee5ed7e229 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:21.300206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650388d-ecc2-4dd4-93ad-0e829282eec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:21.300873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650388d-ecc2-4dd4-93ad-0e829282eec5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.301147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650388d-ecc2-4dd4-93ad-0e829282eec5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.301411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3650388d-ecc2-4dd4-93ad-0e829282eec5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.336076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f35ad2-6f76-48af-9aba-ee9c5b0565d1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:21.340449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:21.375829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd48144f-c34c-48d2-8515-04060cd777e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:21.376830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48144f-c34c-48d2-8515-04060cd777e1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.377419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48144f-c34c-48d2-8515-04060cd777e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.378212 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd48144f-c34c-48d2-8515-04060cd777e1 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.379362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48144f-c34c-48d2-8515-04060cd777e1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:21.379870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48144f-c34c-48d2-8515-04060cd777e1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:21.381581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd48144f-c34c-48d2-8515-04060cd777e1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:21.385404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650388d-ecc2-4dd4-93ad-0e829282eec5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:21.387158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3650388d-ecc2-4dd4-93ad-0e829282eec5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:21.395786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 303/630] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:21.397296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b4333-d573-469d-8382-d046f37a8995 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.397770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b4333-d573-469d-8382-d046f37a8995 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.398247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b4333-d573-469d-8382-d046f37a8995 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.419830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b56b4333-d573-469d-8382-d046f37a8995 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:21.421268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:21.421523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b56b4333-d573-469d-8382-d046f37a8995 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:21.503103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 304/631] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:21.505034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b4333-d573-469d-8382-d046f37a8995 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:21.522539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 328/632] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:21.522539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650388d-ecc2-4dd4-93ad-0e829282eec5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:21.622159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:21.629164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b4333-d573-469d-8382-d046f37a8995 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:21.629753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3650388d-ecc2-4dd4-93ad-0e829282eec5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:21.659591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:21.665736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f43551-73c7-45a3-a4c8-13f2b70964a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:21.666486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f43551-73c7-45a3-a4c8-13f2b70964a4 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.666712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f43551-73c7-45a3-a4c8-13f2b70964a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.666990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f43551-73c7-45a3-a4c8-13f2b70964a4 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/4815ef133a3a4604bc467c222d450fbf/roles/df32bbf3791f4028b2dabf497d5e938d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.672004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ba3fb-f7e3-40c2-939c-c6b812ffa450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:21.672876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ba3fb-f7e3-40c2-939c-c6b812ffa450 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.673365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ba3fb-f7e3-40c2-939c-c6b812ffa450 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.673829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ba3fb-f7e3-40c2-939c-c6b812ffa450 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.681512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f43551-73c7-45a3-a4c8-13f2b70964a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=4815ef133a3a4604bc467c222d450fbf, role_id=df32bbf3791f4028b2dabf497d5e938d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:21.683348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f43551-73c7-45a3-a4c8-13f2b70964a4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:21.686908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324ba3fb-f7e3-40c2-939c-c6b812ffa450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-324ba3fb-f7e3-40c2-939c-c6b812ffa450 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:21.692133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:21.693259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:21.694275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:21.696662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 329/633] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:21.697643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ba3fb-f7e3-40c2-939c-c6b812ffa450 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:21.703796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 305/634] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:48:21 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/4815ef133a3a4604bc467c222d450fbf/roles/df32bbf3791f4028b2dabf497d5e938d => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:48:21.705044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504b02-800e-457c-9d3d-6fae2c4d57ef None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.705331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504b02-800e-457c-9d3d-6fae2c4d57ef None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.705611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504b02-800e-457c-9d3d-6fae2c4d57ef None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.711213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 306/635] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:21.711213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504b02-800e-457c-9d3d-6fae2c4d57ef None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:21.830963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324ba3fb-f7e3-40c2-939c-c6b812ffa450 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:21.836377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504b02-800e-457c-9d3d-6fae2c4d57ef None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:21.870582 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaba80e-5a4c-4a26-b444-a5f0545eeda0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:21.871285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaba80e-5a4c-4a26-b444-a5f0545eeda0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.871572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaba80e-5a4c-4a26-b444-a5f0545eeda0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.871840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaba80e-5a4c-4a26-b444-a5f0545eeda0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.873373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63aa5648-1415-40c4-bfeb-4fa03e5c93d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:21.874000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa5648-1415-40c4-bfeb-4fa03e5c93d6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.874307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa5648-1415-40c4-bfeb-4fa03e5c93d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.874648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63aa5648-1415-40c4-bfeb-4fa03e5c93d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.875320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa5648-1415-40c4-bfeb-4fa03e5c93d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:21.875628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa5648-1415-40c4-bfeb-4fa03e5c93d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:21.877033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63aa5648-1415-40c4-bfeb-4fa03e5c93d6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:21.883603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaba80e-5a4c-4a26-b444-a5f0545eeda0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:21.885106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaba80e-5a4c-4a26-b444-a5f0545eeda0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:21.895458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 307/636] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:21.895458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c426139-1201-45a6-971b-cd7f8649df9a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:21.895458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c426139-1201-45a6-971b-cd7f8649df9a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:21.895458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c426139-1201-45a6-971b-cd7f8649df9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fcaba80e-5a4c-4a26-b444-a5f0545eeda0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:21.904961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:21.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:21.906943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:21.906943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:21.906943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:21.906943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 330/637] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:21.908541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaba80e-5a4c-4a26-b444-a5f0545eeda0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:21.916757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:21.923492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6c426139-1201-45a6-971b-cd7f8649df9a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:21.925863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6c426139-1201-45a6-971b-cd7f8649df9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:21.953342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:22.014111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 308/638] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:22.014111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c426139-1201-45a6-971b-cd7f8649df9a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.031336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaba80e-5a4c-4a26-b444-a5f0545eeda0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.070834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79144e-4a5a-4cbe-b3bc-5525c8ccfb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.071646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79144e-4a5a-4cbe-b3bc-5525c8ccfb76 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.072487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79144e-4a5a-4cbe-b3bc-5525c8ccfb76 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.072829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f79144e-4a5a-4cbe-b3bc-5525c8ccfb76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.087348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f79144e-4a5a-4cbe-b3bc-5525c8ccfb76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9f79144e-4a5a-4cbe-b3bc-5525c8ccfb76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:22.094234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:22.095405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:22.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:22.098267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 331/639] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:22.099180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79144e-4a5a-4cbe-b3bc-5525c8ccfb76 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.134244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c426139-1201-45a6-971b-cd7f8649df9a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.176496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce187451-763e-4c3f-9389-c8b035aa35ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.177566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce187451-763e-4c3f-9389-c8b035aa35ad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.178104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce187451-763e-4c3f-9389-c8b035aa35ad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.178414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce187451-763e-4c3f-9389-c8b035aa35ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.182183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:22.193109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce187451-763e-4c3f-9389-c8b035aa35ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:22.194641 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce187451-763e-4c3f-9389-c8b035aa35ad None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ce187451-763e-4c3f-9389-c8b035aa35ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:22.213545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:22.214791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:22.215786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:22.215786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:22.215786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:22.215786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 309/640] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:22.216022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce187451-763e-4c3f-9389-c8b035aa35ad None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.225668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:22.235891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f79144e-4a5a-4cbe-b3bc-5525c8ccfb76 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.283532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435bc67-2c22-4761-a8ac-d69b4ce86e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.285231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435bc67-2c22-4761-a8ac-d69b4ce86e07 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.285708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435bc67-2c22-4761-a8ac-d69b4ce86e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.286438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b435bc67-2c22-4761-a8ac-d69b4ce86e07 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.287869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435bc67-2c22-4761-a8ac-d69b4ce86e07 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:22.288626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435bc67-2c22-4761-a8ac-d69b4ce86e07 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:22.290522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b435bc67-2c22-4761-a8ac-d69b4ce86e07 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:22.308275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 332/641] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:22.310317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435bc67-2c22-4761-a8ac-d69b4ce86e07 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.336672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce187451-763e-4c3f-9389-c8b035aa35ad None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.375902 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1b1e4-2440-48a0-a2e9-696302fe5d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.376513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1b1e4-2440-48a0-a2e9-696302fe5d69 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.376790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1b1e4-2440-48a0-a2e9-696302fe5d69 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.377062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be1b1e4-2440-48a0-a2e9-696302fe5d69 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.377708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1b1e4-2440-48a0-a2e9-696302fe5d69 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:22.378024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1b1e4-2440-48a0-a2e9-696302fe5d69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:22.379439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be1b1e4-2440-48a0-a2e9-696302fe5d69 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:22.392730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 310/642] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:22.392730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1b1e4-2440-48a0-a2e9-696302fe5d69 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.414755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b435bc67-2c22-4761-a8ac-d69b4ce86e07 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.448185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:22.454762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e4ce9-5c82-47e3-a11c-23ca9b8963a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.455398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e4ce9-5c82-47e3-a11c-23ca9b8963a4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.455679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e4ce9-5c82-47e3-a11c-23ca9b8963a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.455955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2e4ce9-5c82-47e3-a11c-23ca9b8963a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.456610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e4ce9-5c82-47e3-a11c-23ca9b8963a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:22.456916 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e4ce9-5c82-47e3-a11c-23ca9b8963a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:22.458344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2e4ce9-5c82-47e3-a11c-23ca9b8963a4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:22.474625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 333/643] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:22.475513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e4ce9-5c82-47e3-a11c-23ca9b8963a4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.493992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:22.505293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be1b1e4-2440-48a0-a2e9-696302fe5d69 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.541248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb9ce5-8a87-453a-8f6e-949314cb6c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.542021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb9ce5-8a87-453a-8f6e-949314cb6c24 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.542333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb9ce5-8a87-453a-8f6e-949314cb6c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.542609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cb9ce5-8a87-453a-8f6e-949314cb6c24 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.553459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cb9ce5-8a87-453a-8f6e-949314cb6c24 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c2cb9ce5-8a87-453a-8f6e-949314cb6c24 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:22.559582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:22.560720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 311/644] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:22.561556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb9ce5-8a87-453a-8f6e-949314cb6c24 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.588090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2e4ce9-5c82-47e3-a11c-23ca9b8963a4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.632278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8b789-d9e9-47af-9793-782b7e1b5f31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.633304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8b789-d9e9-47af-9793-782b7e1b5f31 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.633587 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8b789-d9e9-47af-9793-782b7e1b5f31 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.634009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8b789-d9e9-47af-9793-782b7e1b5f31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.643285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8b789-d9e9-47af-9793-782b7e1b5f31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:22.645100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b8b789-d9e9-47af-9793-782b7e1b5f31 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-68b8b789-d9e9-47af-9793-782b7e1b5f31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:22.659554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:22.660446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:22.661516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:22.661516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:22.661516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:22.661516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:22.661516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:22.661516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:22.661516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:22.661516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:22.662452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 334/645] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:22.663032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8b789-d9e9-47af-9793-782b7e1b5f31 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.686594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cb9ce5-8a87-453a-8f6e-949314cb6c24 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.697616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:22.726944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33749d01-34ca-4c40-aeea-0e9c40bf3019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.727825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749d01-34ca-4c40-aeea-0e9c40bf3019 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.728156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749d01-34ca-4c40-aeea-0e9c40bf3019 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.728424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33749d01-34ca-4c40-aeea-0e9c40bf3019 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.739553 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33749d01-34ca-4c40-aeea-0e9c40bf3019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:22.741022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33749d01-34ca-4c40-aeea-0e9c40bf3019 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:22.742978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-33749d01-34ca-4c40-aeea-0e9c40bf3019 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:22.756315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:22.757467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:22.758417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:22.758417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:22.758417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:22.758417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 312/646] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:22.758417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33749d01-34ca-4c40-aeea-0e9c40bf3019 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.774558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b8b789-d9e9-47af-9793-782b7e1b5f31 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.811550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af07091-aff8-4994-a912-a6713f391886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.812214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af07091-aff8-4994-a912-a6713f391886 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.812480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af07091-aff8-4994-a912-a6713f391886 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.812761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af07091-aff8-4994-a912-a6713f391886 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.820560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af07091-aff8-4994-a912-a6713f391886 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:22.822043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af07091-aff8-4994-a912-a6713f391886 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3af07091-aff8-4994-a912-a6713f391886 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:22.833470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:22.835060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 13 06:48:22.836225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:22.839361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 335/647] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:22.840337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af07091-aff8-4994-a912-a6713f391886 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.870887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33749d01-34ca-4c40-aeea-0e9c40bf3019 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.906239 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b617f2f-4fa4-49b2-bf31-cefbab5c1aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.906789 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b617f2f-4fa4-49b2-bf31-cefbab5c1aeb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.907077 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b617f2f-4fa4-49b2-bf31-cefbab5c1aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.907354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b617f2f-4fa4-49b2-bf31-cefbab5c1aeb None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.908025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b617f2f-4fa4-49b2-bf31-cefbab5c1aeb None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:22.908338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b617f2f-4fa4-49b2-bf31-cefbab5c1aeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:22.909727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b617f2f-4fa4-49b2-bf31-cefbab5c1aeb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:22.918481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 313/648] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:22.919307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b617f2f-4fa4-49b2-bf31-cefbab5c1aeb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:22.947935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:22.950419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af07091-aff8-4994-a912-a6713f391886 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:22.987593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa0e1-a0b3-47d8-bf5d-ee44f1b4fca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:22.988216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa0e1-a0b3-47d8-bf5d-ee44f1b4fca4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:22.988494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa0e1-a0b3-47d8-bf5d-ee44f1b4fca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:22.988766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54aa0e1-a0b3-47d8-bf5d-ee44f1b4fca4 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:22.989456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa0e1-a0b3-47d8-bf5d-ee44f1b4fca4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:22.989752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa0e1-a0b3-47d8-bf5d-ee44f1b4fca4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:22.991245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54aa0e1-a0b3-47d8-bf5d-ee44f1b4fca4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:22.994180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:23.010887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 336/649] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:23.010887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa0e1-a0b3-47d8-bf5d-ee44f1b4fca4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.039938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b617f2f-4fa4-49b2-bf31-cefbab5c1aeb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:23.078974 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b7e1c-680e-41c0-8d99-cde73b097fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:23.079817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b7e1c-680e-41c0-8d99-cde73b097fb2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.080266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b7e1c-680e-41c0-8d99-cde73b097fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.080705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890b7e1c-680e-41c0-8d99-cde73b097fb2 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.081356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b7e1c-680e-41c0-8d99-cde73b097fb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:23.081661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b7e1c-680e-41c0-8d99-cde73b097fb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:23.083111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890b7e1c-680e-41c0-8d99-cde73b097fb2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:23.097613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 314/650] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:23.099016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b7e1c-680e-41c0-8d99-cde73b097fb2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.120528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54aa0e1-a0b3-47d8-bf5d-ee44f1b4fca4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:23.159304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d879f4a-2741-469f-a863-b121fa59ac67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:23.160228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879f4a-2741-469f-a863-b121fa59ac67 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.160598 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879f4a-2741-469f-a863-b121fa59ac67 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.160875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d879f4a-2741-469f-a863-b121fa59ac67 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.161592 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d879f4a-2741-469f-a863-b121fa59ac67 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:23.161913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d879f4a-2741-469f-a863-b121fa59ac67 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:23.163532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d879f4a-2741-469f-a863-b121fa59ac67 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:23.181076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 337/651] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:23.182508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d879f4a-2741-469f-a863-b121fa59ac67 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.195385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:23.212522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890b7e1c-680e-41c0-8d99-cde73b097fb2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:23.243346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:23.249162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff12530-730e-4f94-8fd5-f5ca4a1c2555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:23.249742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff12530-730e-4f94-8fd5-f5ca4a1c2555 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.250022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff12530-730e-4f94-8fd5-f5ca4a1c2555 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.250330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff12530-730e-4f94-8fd5-f5ca4a1c2555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.260467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff12530-730e-4f94-8fd5-f5ca4a1c2555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:23.262019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff12530-730e-4f94-8fd5-f5ca4a1c2555 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4ff12530-730e-4f94-8fd5-f5ca4a1c2555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:23.277453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:23.278478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:23.279380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:23.279380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:23.279380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:23.279380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 315/652] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:23.279380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff12530-730e-4f94-8fd5-f5ca4a1c2555 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.294506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d879f4a-2741-469f-a863-b121fa59ac67 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:23.331310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a36f27-8ff7-46b6-9bfb-22cce88c5f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:23.332149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a36f27-8ff7-46b6-9bfb-22cce88c5f09 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.332593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a36f27-8ff7-46b6-9bfb-22cce88c5f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.333026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a36f27-8ff7-46b6-9bfb-22cce88c5f09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.333896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a36f27-8ff7-46b6-9bfb-22cce88c5f09 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=87f3a9ffb88f4faa85edc3dabce0c8f5) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:23.334389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a36f27-8ff7-46b6-9bfb-22cce88c5f09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:23.335817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a36f27-8ff7-46b6-9bfb-22cce88c5f09 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:23.341850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 338/653] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/role_assignments?user.id=87f3a9ffb88f4faa85edc3dabce0c8f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:23.342500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a36f27-8ff7-46b6-9bfb-22cce88c5f09 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.425012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff12530-730e-4f94-8fd5-f5ca4a1c2555 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:23.453271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a36f27-8ff7-46b6-9bfb-22cce88c5f09 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:23.473190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:23.474463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951336c-ad3c-4afb-ab98-ab8e8cf65ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:23.475220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951336c-ad3c-4afb-ab98-ab8e8cf65ddb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.475496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951336c-ad3c-4afb-ab98-ab8e8cf65ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.478514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951336c-ad3c-4afb-ab98-ab8e8cf65ddb None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.484233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:23.492946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951336c-ad3c-4afb-ab98-ab8e8cf65ddb None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:23.494822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951336c-ad3c-4afb-ab98-ab8e8cf65ddb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:23.495719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea28ac-af72-4ed7-aa6f-ace19d283a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:23.496545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea28ac-af72-4ed7-aa6f-ace19d283a58 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.496823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea28ac-af72-4ed7-aa6f-ace19d283a58 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.497093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aea28ac-af72-4ed7-aa6f-ace19d283a58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6951336c-ad3c-4afb-ab98-ab8e8cf65ddb None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:23.511362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:23.512464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 06:48:23.513410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:23.516078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 316/654] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/users/cinder => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:23.517937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951336c-ad3c-4afb-ab98-ab8e8cf65ddb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.584470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea28ac-af72-4ed7-aa6f-ace19d283a58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:23.584806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aea28ac-af72-4ed7-aa6f-ace19d283a58 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:23.655051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951336c-ad3c-4afb-ab98-ab8e8cf65ddb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:23.698869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 339/655] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:23.699997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea28ac-af72-4ed7-aa6f-ace19d283a58 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.701096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f016b8-df64-4aee-b252-4922ac227d6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:23.701960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f016b8-df64-4aee-b252-4922ac227d6c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.702751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f016b8-df64-4aee-b252-4922ac227d6c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.704235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f016b8-df64-4aee-b252-4922ac227d6c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.705084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f016b8-df64-4aee-b252-4922ac227d6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:23.705575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f016b8-df64-4aee-b252-4922ac227d6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:23.707184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f016b8-df64-4aee-b252-4922ac227d6c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:23.720929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 317/656] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:23.721873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f016b8-df64-4aee-b252-4922ac227d6c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.723510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:23.797348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:23.831336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aea28ac-af72-4ed7-aa6f-ace19d283a58 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:23.849161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f016b8-df64-4aee-b252-4922ac227d6c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:23.876998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca3a4b-4f51-49f0-a965-9e5df1e23245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:23.878536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3a4b-4f51-49f0-a965-9e5df1e23245 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.879194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3a4b-4f51-49f0-a965-9e5df1e23245 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.879611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aca3a4b-4f51-49f0-a965-9e5df1e23245 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.881128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca3a4b-4f51-49f0-a965-9e5df1e23245 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:23.881612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca3a4b-4f51-49f0-a965-9e5df1e23245 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:23.884046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aca3a4b-4f51-49f0-a965-9e5df1e23245 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:23.888271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ddc6f6-ba88-4117-8a10-ec74fbb90bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:23.889030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc6f6-ba88-4117-8a10-ec74fbb90bca None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.889401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc6f6-ba88-4117-8a10-ec74fbb90bca None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.889763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ddc6f6-ba88-4117-8a10-ec74fbb90bca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.900937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddc6f6-ba88-4117-8a10-ec74fbb90bca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:23.902636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ddc6f6-ba88-4117-8a10-ec74fbb90bca None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:23.906904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 340/657] 10.222.0.25 () {56 vars in 1137 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:23.907934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca3a4b-4f51-49f0-a965-9e5df1e23245 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-03ddc6f6-ba88-4117-8a10-ec74fbb90bca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:23.919582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:23.920787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:23.922107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:23.922107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:23.922107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:23.922107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 318/658] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:23.922107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e630c6-e91a-44b6-ab3b-eb70336a3d0c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:23.922107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e630c6-e91a-44b6-ab3b-eb70336a3d0c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:23.922107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e630c6-e91a-44b6-ab3b-eb70336a3d0c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:23.923460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 319/659] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:23 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:23.923978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e630c6-e91a-44b6-ab3b-eb70336a3d0c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:23.998694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:24.023939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aca3a4b-4f51-49f0-a965-9e5df1e23245 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.037897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e630c6-e91a-44b6-ab3b-eb70336a3d0c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.047880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:24.064163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91975210-53b7-409e-85c0-7221177888ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:24.064804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91975210-53b7-409e-85c0-7221177888ca None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.065081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91975210-53b7-409e-85c0-7221177888ca None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.065355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91975210-53b7-409e-85c0-7221177888ca None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:24.072767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91975210-53b7-409e-85c0-7221177888ca None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:24.074163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c88e9-2961-4507-a380-b61e8df4aaac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:24.074366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91975210-53b7-409e-85c0-7221177888ca None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:24.075028 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c88e9-2961-4507-a380-b61e8df4aaac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.075309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c88e9-2961-4507-a380-b61e8df4aaac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.075587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372c88e9-2961-4507-a380-b61e8df4aaac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-91975210-53b7-409e-85c0-7221177888ca None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:24.086381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:24.087349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 341/660] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:24.089056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91975210-53b7-409e-85c0-7221177888ca None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:24.151462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c88e9-2961-4507-a380-b61e8df4aaac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:24.153037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372c88e9-2961-4507-a380-b61e8df4aaac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:24.196087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91975210-53b7-409e-85c0-7221177888ca None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.232739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fef1887-0e46-4b65-9b07-1087a90c86fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:24.233361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef1887-0e46-4b65-9b07-1087a90c86fa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.233644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef1887-0e46-4b65-9b07-1087a90c86fa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.233923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fef1887-0e46-4b65-9b07-1087a90c86fa None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:24.234602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef1887-0e46-4b65-9b07-1087a90c86fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:24.234905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef1887-0e46-4b65-9b07-1087a90c86fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:24.236334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fef1887-0e46-4b65-9b07-1087a90c86fa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:24.238824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:24.253530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 342/661] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:24.253530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452901-eea7-4ed3-bf86-e7805a88e8db None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.253530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452901-eea7-4ed3-bf86-e7805a88e8db None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.253530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae452901-eea7-4ed3-bf86-e7805a88e8db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:24.268437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 320/662] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:24.269326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c88e9-2961-4507-a380-b61e8df4aaac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:24.269841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ae452901-eea7-4ed3-bf86-e7805a88e8db None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:24.271103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ae452901-eea7-4ed3-bf86-e7805a88e8db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:24.279924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:24.340494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 343/663] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:24.341354 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452901-eea7-4ed3-bf86-e7805a88e8db None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:24.390012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372c88e9-2961-4507-a380-b61e8df4aaac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.437795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af030e-0d53-4916-b6a4-afc79e2e2185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:24.439358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af030e-0d53-4916-b6a4-afc79e2e2185 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.440097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af030e-0d53-4916-b6a4-afc79e2e2185 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.440672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38af030e-0d53-4916-b6a4-afc79e2e2185 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:24.442291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af030e-0d53-4916-b6a4-afc79e2e2185 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:24.442807 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af030e-0d53-4916-b6a4-afc79e2e2185 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:24.444627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38af030e-0d53-4916-b6a4-afc79e2e2185 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:24.461663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae452901-eea7-4ed3-bf86-e7805a88e8db None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.468779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 321/664] 10.222.0.25 () {56 vars in 1139 bytes} [Tue Sep 13 06:48:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:24.469878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af030e-0d53-4916-b6a4-afc79e2e2185 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:24.493226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:24.503743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211150-65d7-4380-8af9-f1be0c1c62d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:24.504367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211150-65d7-4380-8af9-f1be0c1c62d3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.504583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211150-65d7-4380-8af9-f1be0c1c62d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.504853 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df211150-65d7-4380-8af9-f1be0c1c62d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:24.511746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211150-65d7-4380-8af9-f1be0c1c62d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:24.513335 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df211150-65d7-4380-8af9-f1be0c1c62d3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-df211150-65d7-4380-8af9-f1be0c1c62d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:24.533743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:24.534693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:24.535583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:24.535583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:24.535583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:24.535583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:24.535583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:24.535583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:24.535583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:24.535583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:24.542222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 344/665] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:24.542222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211150-65d7-4380-8af9-f1be0c1c62d3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:24.552445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:24.600886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38af030e-0d53-4916-b6a4-afc79e2e2185 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.643321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157a1df-a994-455e-b15c-4b4295621c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:24.644137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157a1df-a994-455e-b15c-4b4295621c88 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.644413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157a1df-a994-455e-b15c-4b4295621c88 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.644679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e157a1df-a994-455e-b15c-4b4295621c88 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:24.655365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157a1df-a994-455e-b15c-4b4295621c88 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:24.656826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e157a1df-a994-455e-b15c-4b4295621c88 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:24.660642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df211150-65d7-4380-8af9-f1be0c1c62d3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e157a1df-a994-455e-b15c-4b4295621c88 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:24.672532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:24.673522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:24.674469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:24.674469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:24.674469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:24.678259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 322/666] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:24.679213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157a1df-a994-455e-b15c-4b4295621c88 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:24.697916 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a2257-6584-483c-9300-f5042786b812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:24.698566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a2257-6584-483c-9300-f5042786b812 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.698848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a2257-6584-483c-9300-f5042786b812 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.699111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52a2257-6584-483c-9300-f5042786b812 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:24.755499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:24.770688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a2257-6584-483c-9300-f5042786b812 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:24.772161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52a2257-6584-483c-9300-f5042786b812 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:24.791607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e157a1df-a994-455e-b15c-4b4295621c88 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.810966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:24.828966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b2206-fa60-4569-bf4e-47d469d8ddd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:24.829599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b2206-fa60-4569-bf4e-47d469d8ddd2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.829881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b2206-fa60-4569-bf4e-47d469d8ddd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.830189 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84b2206-fa60-4569-bf4e-47d469d8ddd2 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:24.830895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b2206-fa60-4569-bf4e-47d469d8ddd2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:24.831208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b2206-fa60-4569-bf4e-47d469d8ddd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:24.832659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84b2206-fa60-4569-bf4e-47d469d8ddd2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:24.844528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 323/667] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:24.845330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b2206-fa60-4569-bf4e-47d469d8ddd2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:24.890717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 345/668] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:24.890717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a2257-6584-483c-9300-f5042786b812 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:24.952183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84b2206-fa60-4569-bf4e-47d469d8ddd2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.977131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:24.992097 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52a2257-6584-483c-9300-f5042786b812 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:24.992444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22feaca-4d2c-483e-83b9-cfdd8d78d171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:24.993352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22feaca-4d2c-483e-83b9-cfdd8d78d171 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:24.993935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22feaca-4d2c-483e-83b9-cfdd8d78d171 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:24.994519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22feaca-4d2c-483e-83b9-cfdd8d78d171 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.009492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22feaca-4d2c-483e-83b9-cfdd8d78d171 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b22feaca-4d2c-483e-83b9-cfdd8d78d171 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:25.014379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:25.015312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:25.016152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 324/669] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:25.016839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22feaca-4d2c-483e-83b9-cfdd8d78d171 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.030371 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91302d8e-d006-4e01-8e43-04cbddd4304c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.030985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91302d8e-d006-4e01-8e43-04cbddd4304c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.031261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91302d8e-d006-4e01-8e43-04cbddd4304c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.031778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91302d8e-d006-4e01-8e43-04cbddd4304c None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.032162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91302d8e-d006-4e01-8e43-04cbddd4304c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:25.032460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91302d8e-d006-4e01-8e43-04cbddd4304c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.033831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91302d8e-d006-4e01-8e43-04cbddd4304c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.045771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 346/670] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:25.046688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91302d8e-d006-4e01-8e43-04cbddd4304c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.067906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:25.130054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22feaca-4d2c-483e-83b9-cfdd8d78d171 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:25.147163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91302d8e-d006-4e01-8e43-04cbddd4304c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:25.166054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630eb53-7202-41f0-adcb-205bc967e294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.166699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630eb53-7202-41f0-adcb-205bc967e294 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.166985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630eb53-7202-41f0-adcb-205bc967e294 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.167256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f630eb53-7202-41f0-adcb-205bc967e294 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.168029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630eb53-7202-41f0-adcb-205bc967e294 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=3505c8965c4a42e493a2f5916cab56dc) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:25.168467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630eb53-7202-41f0-adcb-205bc967e294 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.170037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f630eb53-7202-41f0-adcb-205bc967e294 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.176309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 325/671] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/role_assignments?user.id=3505c8965c4a42e493a2f5916cab56dc&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:25.177118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630eb53-7202-41f0-adcb-205bc967e294 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.184559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509d57d-7815-42aa-9c6d-13bcc0e7f351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.185642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509d57d-7815-42aa-9c6d-13bcc0e7f351 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.186162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509d57d-7815-42aa-9c6d-13bcc0e7f351 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.186642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3509d57d-7815-42aa-9c6d-13bcc0e7f351 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.207667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509d57d-7815-42aa-9c6d-13bcc0e7f351 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.210272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3509d57d-7815-42aa-9c6d-13bcc0e7f351 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.222794 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3509d57d-7815-42aa-9c6d-13bcc0e7f351 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:25.235978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:25.237006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:25.237884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:25.237884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:25.237884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:25.241233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 347/672] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:25.242466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509d57d-7815-42aa-9c6d-13bcc0e7f351 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.314038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f630eb53-7202-41f0-adcb-205bc967e294 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:25.316241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:25.361549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda31427-b6d1-4678-9e00-0d69ec3bbef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.362240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda31427-b6d1-4678-9e00-0d69ec3bbef7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.362530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda31427-b6d1-4678-9e00-0d69ec3bbef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.362811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda31427-b6d1-4678-9e00-0d69ec3bbef7 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.363487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda31427-b6d1-4678-9e00-0d69ec3bbef7 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:25.363796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda31427-b6d1-4678-9e00-0d69ec3bbef7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.365978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda31427-b6d1-4678-9e00-0d69ec3bbef7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.375894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 326/673] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:25.376818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda31427-b6d1-4678-9e00-0d69ec3bbef7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.383879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3509d57d-7815-42aa-9c6d-13bcc0e7f351 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:25.424699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874d06-0d2e-4a83-9010-3817b5ea4e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.425343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874d06-0d2e-4a83-9010-3817b5ea4e31 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.425629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874d06-0d2e-4a83-9010-3817b5ea4e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.425901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e874d06-0d2e-4a83-9010-3817b5ea4e31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.435184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e874d06-0d2e-4a83-9010-3817b5ea4e31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.436661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e874d06-0d2e-4a83-9010-3817b5ea4e31 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3e874d06-0d2e-4a83-9010-3817b5ea4e31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:25.449382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:25.450465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:25.451398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:25.451398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:25.451398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:25.451398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:25.451398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:25.451398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:25.451398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:25.451398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:25.451398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 348/674] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:25.451902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874d06-0d2e-4a83-9010-3817b5ea4e31 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.511016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda31427-b6d1-4678-9e00-0d69ec3bbef7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:25.544015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:25.547138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81797713-e253-4fe5-88a0-5b5842614efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.547783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81797713-e253-4fe5-88a0-5b5842614efd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.548063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81797713-e253-4fe5-88a0-5b5842614efd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.548339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81797713-e253-4fe5-88a0-5b5842614efd None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.549158 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81797713-e253-4fe5-88a0-5b5842614efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:25.549319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81797713-e253-4fe5-88a0-5b5842614efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.550797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81797713-e253-4fe5-88a0-5b5842614efd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.570333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 327/675] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:25.570333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81797713-e253-4fe5-88a0-5b5842614efd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.597784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:25.598305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e874d06-0d2e-4a83-9010-3817b5ea4e31 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:25.642285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67399ba-ee68-46af-8185-3bf36d6535a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.643333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67399ba-ee68-46af-8185-3bf36d6535a3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.643814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67399ba-ee68-46af-8185-3bf36d6535a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.644350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67399ba-ee68-46af-8185-3bf36d6535a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.657140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67399ba-ee68-46af-8185-3bf36d6535a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.659005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67399ba-ee68-46af-8185-3bf36d6535a3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c67399ba-ee68-46af-8185-3bf36d6535a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:25.676021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:25.677022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:25.677925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:25.677925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:25.677925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:25.679502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 349/676] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:25.679703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81797713-e253-4fe5-88a0-5b5842614efd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:25.680668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67399ba-ee68-46af-8185-3bf36d6535a3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.716034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbccfe-3811-4f54-bdb8-0b4ee2aa78ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.716696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbccfe-3811-4f54-bdb8-0b4ee2aa78ea None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.716930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbccfe-3811-4f54-bdb8-0b4ee2aa78ea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.717262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbccfe-3811-4f54-bdb8-0b4ee2aa78ea None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.717965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbccfe-3811-4f54-bdb8-0b4ee2aa78ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:25.718304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbccfe-3811-4f54-bdb8-0b4ee2aa78ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.719777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbccfe-3811-4f54-bdb8-0b4ee2aa78ea None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.732082 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 328/677] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:25.733309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cad3d-66f6-4e8f-907f-ddb4f4318561 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.733599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cad3d-66f6-4e8f-907f-ddb4f4318561 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.733876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5cad3d-66f6-4e8f-907f-ddb4f4318561 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.735234 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 329/678] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:25.735781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cad3d-66f6-4e8f-907f-ddb4f4318561 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.801378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67399ba-ee68-46af-8185-3bf36d6535a3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:25.815097 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:25.828610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:25.843011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc7ab8-758a-40ef-bef7-85b070839998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.843951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc7ab8-758a-40ef-bef7-85b070839998 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.844231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc7ab8-758a-40ef-bef7-85b070839998 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.844509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddc7ab8-758a-40ef-bef7-85b070839998 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.847354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5cad3d-66f6-4e8f-907f-ddb4f4318561 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:25.851742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc7ab8-758a-40ef-bef7-85b070839998 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.853204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddc7ab8-758a-40ef-bef7-85b070839998 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0ddc7ab8-758a-40ef-bef7-85b070839998 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:25.865009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:25.866142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:25.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:25.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:25.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:25.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:25.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:25.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:25.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:25.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:25.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 350/679] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:25.867550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc7ab8-758a-40ef-bef7-85b070839998 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:25.891660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5b1d81-0acb-47b3-9883-e940916263fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:25.892280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b1d81-0acb-47b3-9883-e940916263fb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.892556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b1d81-0acb-47b3-9883-e940916263fb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.892824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5b1d81-0acb-47b3-9883-e940916263fb None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.893491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b1d81-0acb-47b3-9883-e940916263fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:25.893787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b1d81-0acb-47b3-9883-e940916263fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:25.895201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc5b1d81-0acb-47b3-9883-e940916263fb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:25.907559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 330/680] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:25.908758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d807f6-0731-4644-a584-a8f373e0974c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:25.909036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d807f6-0731-4644-a584-a8f373e0974c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:25.909306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d807f6-0731-4644-a584-a8f373e0974c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:25.927686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d3d807f6-0731-4644-a584-a8f373e0974c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:25.928869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d3d807f6-0731-4644-a584-a8f373e0974c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:25.984427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddc7ab8-758a-40ef-bef7-85b070839998 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:26.003219 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 331/681] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:26.005283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d807f6-0731-4644-a584-a8f373e0974c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:26.018843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0d786-86ea-44f6-b607-bcf4aec02591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:26.019459 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0d786-86ea-44f6-b607-bcf4aec02591 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:26.019733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0d786-86ea-44f6-b607-bcf4aec02591 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:26.020000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd0d786-86ea-44f6-b607-bcf4aec02591 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:26.020725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0d786-86ea-44f6-b607-bcf4aec02591 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=88520ce6493a44f39b19c67592f03632) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:26.021028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0d786-86ea-44f6-b607-bcf4aec02591 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:26.022507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd0d786-86ea-44f6-b607-bcf4aec02591 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:26.029111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 351/682] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:25 2022] GET /identity/v3/role_assignments?user.id=88520ce6493a44f39b19c67592f03632&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:26.035232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0d786-86ea-44f6-b607-bcf4aec02591 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:26.106309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:26.116060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:26.143080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d807f6-0731-4644-a584-a8f373e0974c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:26.213049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd0d786-86ea-44f6-b607-bcf4aec02591 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:26.254039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba5d6c-eb8e-4348-b1f0-fc9126cd503f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:26.255145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5d6c-eb8e-4348-b1f0-fc9126cd503f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:26.255774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5d6c-eb8e-4348-b1f0-fc9126cd503f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:26.256323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cba5d6c-eb8e-4348-b1f0-fc9126cd503f None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:26.265636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba5d6c-eb8e-4348-b1f0-fc9126cd503f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:26.267557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cba5d6c-eb8e-4348-b1f0-fc9126cd503f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:26.271152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ada0c8-3c0c-490c-8f57-b5d73494e70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:26.271440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ada0c8-3c0c-490c-8f57-b5d73494e70d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:26.271723 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ada0c8-3c0c-490c-8f57-b5d73494e70d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:26.271998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ada0c8-3c0c-490c-8f57-b5d73494e70d None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:26.272712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ada0c8-3c0c-490c-8f57-b5d73494e70d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:26.273018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ada0c8-3c0c-490c-8f57-b5d73494e70d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:26.274481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ada0c8-3c0c-490c-8f57-b5d73494e70d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7cba5d6c-eb8e-4348-b1f0-fc9126cd503f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:26.281662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:26.282902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:26.284070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:26.285715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 352/683] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:26 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:26.293633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 332/684] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:26.295364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba5d6c-eb8e-4348-b1f0-fc9126cd503f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:26.299003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ada0c8-3c0c-490c-8f57-b5d73494e70d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:26.402723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:26.412397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cba5d6c-eb8e-4348-b1f0-fc9126cd503f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:26.451702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb5180-99cb-44c2-9b71-8c7a22a22c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:26.452418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb5180-99cb-44c2-9b71-8c7a22a22c0c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:26.452784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb5180-99cb-44c2-9b71-8c7a22a22c0c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:26.453526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdb5180-99cb-44c2-9b71-8c7a22a22c0c None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:26.454374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb5180-99cb-44c2-9b71-8c7a22a22c0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:26.454757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb5180-99cb-44c2-9b71-8c7a22a22c0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:26.456575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdb5180-99cb-44c2-9b71-8c7a22a22c0c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:26.470174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ada0c8-3c0c-490c-8f57-b5d73494e70d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:26.479308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 353/685] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Sep 13 06:48:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:26.485249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb5180-99cb-44c2-9b71-8c7a22a22c0c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:26.534063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f322a-ce25-4c64-96d3-c6ed4f8532fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:26.535459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f322a-ce25-4c64-96d3-c6ed4f8532fc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:26.536303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f322a-ce25-4c64-96d3-c6ed4f8532fc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:26.536980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248f322a-ce25-4c64-96d3-c6ed4f8532fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:26.538780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f322a-ce25-4c64-96d3-c6ed4f8532fc None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=f94aa8af227a42a08d8a0e4b7f02b85a) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:26.539536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f322a-ce25-4c64-96d3-c6ed4f8532fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:26.544036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248f322a-ce25-4c64-96d3-c6ed4f8532fc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:26.547879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:26.558216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 333/686] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:26 2022] GET /identity/v3/role_assignments?user.id=f94aa8af227a42a08d8a0e4b7f02b85a&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 608 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:26.560002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f322a-ce25-4c64-96d3-c6ed4f8532fc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:26.641526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdb5180-99cb-44c2-9b71-8c7a22a22c0c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:26.720639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38821c33-a3f3-4be7-8983-ec4c590301c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:26.721526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38821c33-a3f3-4be7-8983-ec4c590301c0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:26.721957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38821c33-a3f3-4be7-8983-ec4c590301c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:26.722384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38821c33-a3f3-4be7-8983-ec4c590301c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:26.735548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38821c33-a3f3-4be7-8983-ec4c590301c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-38821c33-a3f3-4be7-8983-ec4c590301c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:26.740685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:26.741727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:26.742641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:26.745229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 354/687] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:26.754549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248f322a-ce25-4c64-96d3-c6ed4f8532fc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:26.755890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38821c33-a3f3-4be7-8983-ec4c590301c0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:26.786293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:26.794359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6eaa97-bc6f-45d7-94a6-50f79ca0eedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:26.795099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eaa97-bc6f-45d7-94a6-50f79ca0eedf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:26.795446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eaa97-bc6f-45d7-94a6-50f79ca0eedf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:26.796673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6eaa97-bc6f-45d7-94a6-50f79ca0eedf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:26.870444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:26.875184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6eaa97-bc6f-45d7-94a6-50f79ca0eedf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:26.876956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6eaa97-bc6f-45d7-94a6-50f79ca0eedf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:26.929849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38821c33-a3f3-4be7-8983-ec4c590301c0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:26.977987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584424c1-fda5-48bf-a0b1-f884d16d8761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:26.978840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584424c1-fda5-48bf-a0b1-f884d16d8761 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:26.979120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584424c1-fda5-48bf-a0b1-f884d16d8761 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:26.979395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584424c1-fda5-48bf-a0b1-f884d16d8761 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:26.980372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584424c1-fda5-48bf-a0b1-f884d16d8761 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:26.981239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584424c1-fda5-48bf-a0b1-f884d16d8761 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:26.982948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584424c1-fda5-48bf-a0b1-f884d16d8761 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:26.995252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 355/688] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:27.002449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584424c1-fda5-48bf-a0b1-f884d16d8761 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:27.004993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 334/689] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:27.010526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6eaa97-bc6f-45d7-94a6-50f79ca0eedf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:27.102338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:27.130666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:27.133530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584424c1-fda5-48bf-a0b1-f884d16d8761 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:27.139661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6eaa97-bc6f-45d7-94a6-50f79ca0eedf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:27.176725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edbf50-c537-4335-b170-308b34612efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:27.177370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edbf50-c537-4335-b170-308b34612efa None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:27.177652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edbf50-c537-4335-b170-308b34612efa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:27.177927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3edbf50-c537-4335-b170-308b34612efa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:27.191510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edbf50-c537-4335-b170-308b34612efa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:27.193351 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3edbf50-c537-4335-b170-308b34612efa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e3edbf50-c537-4335-b170-308b34612efa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:27.210846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:27.213205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:27.215326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:27.215326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:27.215326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:27.215326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 356/690] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:27.215826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da56871-f1a0-40fa-bfa4-2456611d94c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:27.215826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da56871-f1a0-40fa-bfa4-2456611d94c9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:27.215826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da56871-f1a0-40fa-bfa4-2456611d94c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:27.215826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da56871-f1a0-40fa-bfa4-2456611d94c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:27.219146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edbf50-c537-4335-b170-308b34612efa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:27.238985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da56871-f1a0-40fa-bfa4-2456611d94c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:27.240459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da56871-f1a0-40fa-bfa4-2456611d94c9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6da56871-f1a0-40fa-bfa4-2456611d94c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:27.272283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:27.274579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:27.276716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:27.276716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:27.276716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:27.282555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 335/691] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:27.283633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da56871-f1a0-40fa-bfa4-2456611d94c9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:27.331040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3edbf50-c537-4335-b170-308b34612efa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:27.363561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:27.371497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0b809-a79d-4ed1-aa03-39db3af42eda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:27.372751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0b809-a79d-4ed1-aa03-39db3af42eda None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:27.373190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0b809-a79d-4ed1-aa03-39db3af42eda None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:27.373800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0b809-a79d-4ed1-aa03-39db3af42eda None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:27.374394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0b809-a79d-4ed1-aa03-39db3af42eda None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:27.374925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0b809-a79d-4ed1-aa03-39db3af42eda None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:27.376651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc0b809-a79d-4ed1-aa03-39db3af42eda None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:27.391868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 357/692] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:27.398013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0b809-a79d-4ed1-aa03-39db3af42eda None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:27.485776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da56871-f1a0-40fa-bfa4-2456611d94c9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:27.515554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc0b809-a79d-4ed1-aa03-39db3af42eda None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:27.554971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a360da7-9d79-4b41-8856-d7da88844d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:27.555668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a360da7-9d79-4b41-8856-d7da88844d72 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:27.555948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a360da7-9d79-4b41-8856-d7da88844d72 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:27.556268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a360da7-9d79-4b41-8856-d7da88844d72 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:27.560211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ffce1-2501-419d-8cee-9125e9942ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:27.560845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffce1-2501-419d-8cee-9125e9942ef0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:27.561120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffce1-2501-419d-8cee-9125e9942ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:27.561391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10ffce1-2501-419d-8cee-9125e9942ef0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:27.562046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ffce1-2501-419d-8cee-9125e9942ef0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:27.562375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ffce1-2501-419d-8cee-9125e9942ef0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:27.563774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f10ffce1-2501-419d-8cee-9125e9942ef0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:27.571324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a360da7-9d79-4b41-8856-d7da88844d72 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:27.573929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a360da7-9d79-4b41-8856-d7da88844d72 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:27.593253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 336/693] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7a360da7-9d79-4b41-8856-d7da88844d72 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:27.598558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:27.599572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:27.600533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:27.601079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 358/694] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Sep 13 06:48:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:27.603936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ffce1-2501-419d-8cee-9125e9942ef0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:27.605762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a360da7-9d79-4b41-8856-d7da88844d72 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:27.619405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:27.715760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10ffce1-2501-419d-8cee-9125e9942ef0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:27.719148 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:27.794865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a360da7-9d79-4b41-8856-d7da88844d72 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:27.800588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cd923-e772-4625-ba18-3a143acb64c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:27.801269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cd923-e772-4625-ba18-3a143acb64c0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:27.801548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cd923-e772-4625-ba18-3a143acb64c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:27.801821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747cd923-e772-4625-ba18-3a143acb64c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:27.824746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747cd923-e772-4625-ba18-3a143acb64c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:27.826284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747cd923-e772-4625-ba18-3a143acb64c0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:27.843468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d2e25-6431-4176-8d70-e0953c8431e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:27.844261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d2e25-6431-4176-8d70-e0953c8431e6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:27.844640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d2e25-6431-4176-8d70-e0953c8431e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:27.845034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794d2e25-6431-4176-8d70-e0953c8431e6 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:27.845894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d2e25-6431-4176-8d70-e0953c8431e6 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:27.846329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d2e25-6431-4176-8d70-e0953c8431e6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:27.847926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794d2e25-6431-4176-8d70-e0953c8431e6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-747cd923-e772-4625-ba18-3a143acb64c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:27.852227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:27.853369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:27.854510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:27.854510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:27.854510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:27.854510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:27.854510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:27.854510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:27.854510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:27.854510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:27.856360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 337/695] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:27.863145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cd923-e772-4625-ba18-3a143acb64c0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:27.889565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 359/696] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Sep 13 06:48:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:27.892876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d2e25-6431-4176-8d70-e0953c8431e6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:27.975185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:28.013255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747cd923-e772-4625-ba18-3a143acb64c0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:28.021582 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794d2e25-6431-4176-8d70-e0953c8431e6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:28.024799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:28.056690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ea7376-d2f5-4076-9067-b9752759c12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:28.057663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea7376-d2f5-4076-9067-b9752759c12f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.058594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea7376-d2f5-4076-9067-b9752759c12f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.059035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ea7376-d2f5-4076-9067-b9752759c12f None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.059963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea7376-d2f5-4076-9067-b9752759c12f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:28.060427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea7376-d2f5-4076-9067-b9752759c12f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:28.061346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c26d5-518b-48ae-9ed9-cb4a0f8f2ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:28.061964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c26d5-518b-48ae-9ed9-cb4a0f8f2ae9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.062288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c26d5-518b-48ae-9ed9-cb4a0f8f2ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.062561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c26d5-518b-48ae-9ed9-cb4a0f8f2ae9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.062815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ea7376-d2f5-4076-9067-b9752759c12f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:28.083723 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 338/697] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:28.083723 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d879a-8e27-411b-8cd7-e801b4d1cc28 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.084015 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d879a-8e27-411b-8cd7-e801b4d1cc28 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.084507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d879a-8e27-411b-8cd7-e801b4d1cc28 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.086569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 339/698] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:28.087271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d879a-8e27-411b-8cd7-e801b4d1cc28 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:28.142014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c26d5-518b-48ae-9ed9-cb4a0f8f2ae9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:28.143742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c26d5-518b-48ae-9ed9-cb4a0f8f2ae9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:28.205915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d879a-8e27-411b-8cd7-e801b4d1cc28 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:28.245869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:28.253853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b15e0a-a70a-4b26-b58c-5633577e8a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:28.254485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b15e0a-a70a-4b26-b58c-5633577e8a81 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.254764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b15e0a-a70a-4b26-b58c-5633577e8a81 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.255037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b15e0a-a70a-4b26-b58c-5633577e8a81 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.255767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b15e0a-a70a-4b26-b58c-5633577e8a81 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=4815ef133a3a4604bc467c222d450fbf) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:28.256074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b15e0a-a70a-4b26-b58c-5633577e8a81 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:28.257591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 360/699] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:28.257705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b15e0a-a70a-4b26-b58c-5633577e8a81 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:28.259250 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606afe3e-cc4d-4583-87ed-63605538ea59 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.259796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606afe3e-cc4d-4583-87ed-63605538ea59 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.260355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606afe3e-cc4d-4583-87ed-63605538ea59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.269361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 340/700] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:28 2022] GET /identity/v3/role_assignments?user.id=4815ef133a3a4604bc467c222d450fbf&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:28.270353 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b15e0a-a70a-4b26-b58c-5633577e8a81 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:28.271875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:28.284476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-606afe3e-cc4d-4583-87ed-63605538ea59 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:28.287133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-606afe3e-cc4d-4583-87ed-63605538ea59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:28.398478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 361/701] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:28.451150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b15e0a-a70a-4b26-b58c-5633577e8a81 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:28.489533 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e90a868-9be1-483e-b955-cd74cb7d0b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:28.490235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90a868-9be1-483e-b955-cd74cb7d0b91 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.490517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90a868-9be1-483e-b955-cd74cb7d0b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.490844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e90a868-9be1-483e-b955-cd74cb7d0b91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.503809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90a868-9be1-483e-b955-cd74cb7d0b91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:28.505349 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e90a868-9be1-483e-b955-cd74cb7d0b91 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5e90a868-9be1-483e-b955-cd74cb7d0b91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:28.521512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:28.522555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:28.523545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:28.523545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:28.523545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:28.523545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 341/702] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:28.531510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606afe3e-cc4d-4583-87ed-63605538ea59 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:28.577918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d530cfb-ea52-4270-85fb-5a12792fab9a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.578289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d530cfb-ea52-4270-85fb-5a12792fab9a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.578554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d530cfb-ea52-4270-85fb-5a12792fab9a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.580265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 342/703] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:28.585472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f129cc-32dd-4fee-a47b-355d4030a43d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.585760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f129cc-32dd-4fee-a47b-355d4030a43d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.586045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f129cc-32dd-4fee-a47b-355d4030a43d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.615009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-42f129cc-32dd-4fee-a47b-355d4030a43d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:28.616231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-42f129cc-32dd-4fee-a47b-355d4030a43d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:28.657461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606afe3e-cc4d-4583-87ed-63605538ea59 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:28.665088 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:28.695456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e2f79-1b65-4f0a-8f86-540ae63ebf20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:28.696294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e2f79-1b65-4f0a-8f86-540ae63ebf20 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.696664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e2f79-1b65-4f0a-8f86-540ae63ebf20 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.697028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e2f79-1b65-4f0a-8f86-540ae63ebf20 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.697774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e2f79-1b65-4f0a-8f86-540ae63ebf20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:28.698313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e2f79-1b65-4f0a-8f86-540ae63ebf20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:28.698471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 343/704] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:48:28.702176 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e2f79-1b65-4f0a-8f86-540ae63ebf20 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:28.702176 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:28.713262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 362/705] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:28.719330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f129cc-32dd-4fee-a47b-355d4030a43d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:28.742993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e2f79-1b65-4f0a-8f86-540ae63ebf20 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:28.841006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f129cc-32dd-4fee-a47b-355d4030a43d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:28.854585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e2f79-1b65-4f0a-8f86-540ae63ebf20 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:28.879506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af62448-721e-4265-919c-b76f4b749942 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:28.880656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af62448-721e-4265-919c-b76f4b749942 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.881132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af62448-721e-4265-919c-b76f4b749942 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.881645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af62448-721e-4265-919c-b76f4b749942 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.891410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af62448-721e-4265-919c-b76f4b749942 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:28.892798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1ea70-31f7-491a-9e15-a401b8a70f85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:28.893446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1ea70-31f7-491a-9e15-a401b8a70f85 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:28.893724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1ea70-31f7-491a-9e15-a401b8a70f85 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:28.893908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af62448-721e-4265-919c-b76f4b749942 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:28.894016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b1ea70-31f7-491a-9e15-a401b8a70f85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:28.905753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1ea70-31f7-491a-9e15-a401b8a70f85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4af62448-721e-4265-919c-b76f4b749942 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:28.907658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:28.908562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:28.909454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:28.909454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:28.909454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:28.909454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:28.909454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:28.909454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:28.909454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:28.909454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:28.909881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b1ea70-31f7-491a-9e15-a401b8a70f85 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:28.914471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 344/706] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:28.918408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af62448-721e-4265-919c-b76f4b749942 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-34b1ea70-31f7-491a-9e15-a401b8a70f85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:28.923728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:28.924669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:28.927070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:28.927070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:28.927070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:28.927070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 363/707] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:28.933207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1ea70-31f7-491a-9e15-a401b8a70f85 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:28.967451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:28.988349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:29.042329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af62448-721e-4265-919c-b76f4b749942 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:29.047063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b1ea70-31f7-491a-9e15-a401b8a70f85 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:29.093099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9a70a-3256-42b8-ae12-dbf151b01c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:29.093719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9a70a-3256-42b8-ae12-dbf151b01c8f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:29.094000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9a70a-3256-42b8-ae12-dbf151b01c8f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:29.094302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9a70a-3256-42b8-ae12-dbf151b01c8f None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:29.095146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9a70a-3256-42b8-ae12-dbf151b01c8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:29.095319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9a70a-3256-42b8-ae12-dbf151b01c8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:29.096747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b9a70a-3256-42b8-ae12-dbf151b01c8f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:29.101047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91290c8-a977-441a-97b4-ed1f2a724e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:29.101671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91290c8-a977-441a-97b4-ed1f2a724e70 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:29.101953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91290c8-a977-441a-97b4-ed1f2a724e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:29.102270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91290c8-a977-441a-97b4-ed1f2a724e70 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:29.102910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91290c8-a977-441a-97b4-ed1f2a724e70 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:29.103217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91290c8-a977-441a-97b4-ed1f2a724e70 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:29.104588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91290c8-a977-441a-97b4-ed1f2a724e70 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:29.114192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 345/708] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:29.115099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9a70a-3256-42b8-ae12-dbf151b01c8f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:29.117996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 364/709] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:29.118924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91290c8-a977-441a-97b4-ed1f2a724e70 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:29.228123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:29.241452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9a70a-3256-42b8-ae12-dbf151b01c8f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:29.251676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91290c8-a977-441a-97b4-ed1f2a724e70 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:29.261831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:29.287524 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9646-c266-4932-a2c1-d85bf9f91c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:29.288293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9646-c266-4932-a2c1-d85bf9f91c21 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:29.288632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9646-c266-4932-a2c1-d85bf9f91c21 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:29.289002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06fc9646-c266-4932-a2c1-d85bf9f91c21 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:29.289753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc9646-c266-4932-a2c1-d85bf9f91c21 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:29.291526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06fc9646-c266-4932-a2c1-d85bf9f91c21 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:29.294464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589208a-da5d-4951-89dd-5135ab0200c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:29.295348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589208a-da5d-4951-89dd-5135ab0200c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:29.296131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589208a-da5d-4951-89dd-5135ab0200c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:29.296580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f589208a-da5d-4951-89dd-5135ab0200c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:29.297763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589208a-da5d-4951-89dd-5135ab0200c4 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=87f3a9ffb88f4faa85edc3dabce0c8f5) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:29.298284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589208a-da5d-4951-89dd-5135ab0200c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:29.299919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f589208a-da5d-4951-89dd-5135ab0200c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-06fc9646-c266-4932-a2c1-d85bf9f91c21 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:29.305097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:29.306336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 06:48:29.307194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:29.315671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 346/711] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Sep 13 06:48:29 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:29.315671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9646-c266-4932-a2c1-d85bf9f91c21 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:29.316332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 365/711] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:29 2022] GET /identity/v3/role_assignments?user.id=87f3a9ffb88f4faa85edc3dabce0c8f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:29.319393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589208a-da5d-4951-89dd-5135ab0200c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:29.434625 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06fc9646-c266-4932-a2c1-d85bf9f91c21 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:29.472764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f589208a-da5d-4951-89dd-5135ab0200c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:29.475139 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b083dc11-9f35-47f4-85eb-5d3b818c9903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:29.475833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083dc11-9f35-47f4-85eb-5d3b818c9903 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:29.476112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083dc11-9f35-47f4-85eb-5d3b818c9903 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:29.476380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b083dc11-9f35-47f4-85eb-5d3b818c9903 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:29.477040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083dc11-9f35-47f4-85eb-5d3b818c9903 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:29.478692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b083dc11-9f35-47f4-85eb-5d3b818c9903 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:29.505578 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 347/712] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:48:29 2022] POST /identity/v3/users => generated 254 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:29.507232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d8c4-64b7-4906-8aa4-2572ceb4bd49 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:29.507515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d8c4-64b7-4906-8aa4-2572ceb4bd49 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:29.507831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6106d8c4-64b7-4906-8aa4-2572ceb4bd49 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:29.510226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 348/713] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:29.515228 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7888cf0-bdb8-49e9-a856-d39a17728df1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:29.515647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7888cf0-bdb8-49e9-a856-d39a17728df1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:29.516014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7888cf0-bdb8-49e9-a856-d39a17728df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:29.520517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb02f9f-1cfb-4dd0-9db3-bc87d38810ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:29.521150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb02f9f-1cfb-4dd0-9db3-bc87d38810ed None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:29.521426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb02f9f-1cfb-4dd0-9db3-bc87d38810ed None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:29.521744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb02f9f-1cfb-4dd0-9db3-bc87d38810ed None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:29.522432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb02f9f-1cfb-4dd0-9db3-bc87d38810ed None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:29.522732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb02f9f-1cfb-4dd0-9db3-bc87d38810ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:29.524421 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb02f9f-1cfb-4dd0-9db3-bc87d38810ed None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:29.531556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 366/714] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Sep 13 06:48:29 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:29.539300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb02f9f-1cfb-4dd0-9db3-bc87d38810ed None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:29.540793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c7888cf0-bdb8-49e9-a856-d39a17728df1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:29.542173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c7888cf0-bdb8-49e9-a856-d39a17728df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:29.558272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:29.594493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:29.641540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 349/715] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:29.674307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb02f9f-1cfb-4dd0-9db3-bc87d38810ed None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:29.717061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89505ca-b161-402a-bf8b-f8878d2c5be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:29.717692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89505ca-b161-402a-bf8b-f8878d2c5be0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:29.717968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89505ca-b161-402a-bf8b-f8878d2c5be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:29.718314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89505ca-b161-402a-bf8b-f8878d2c5be0 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:29.718948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89505ca-b161-402a-bf8b-f8878d2c5be0 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:29.719246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89505ca-b161-402a-bf8b-f8878d2c5be0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:29.720754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89505ca-b161-402a-bf8b-f8878d2c5be0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:29.727208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 367/716] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Sep 13 06:48:29 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:29.929976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7888cf0-bdb8-49e9-a856-d39a17728df1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:30.058650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7888cf0-bdb8-49e9-a856-d39a17728df1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:30.109594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc1d8b-1f82-4a52-bccd-38ff455ac47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:30.110307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc1d8b-1f82-4a52-bccd-38ff455ac47c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:30.110585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc1d8b-1f82-4a52-bccd-38ff455ac47c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:30.110908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcc1d8b-1f82-4a52-bccd-38ff455ac47c None admin] PATH_INFO: `/v3/services/network` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:30.111503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc1d8b-1f82-4a52-bccd-38ff455ac47c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:30.112875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebcc1d8b-1f82-4a52-bccd-38ff455ac47c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ebcc1d8b-1f82-4a52-bccd-38ff455ac47c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:30.127512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:30.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:30.129967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 350/717] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:29 2022] GET /identity/v3/services/network => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:30.134445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89505ca-b161-402a-bf8b-f8878d2c5be0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:30.296549 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:30.305838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89505ca-b161-402a-bf8b-f8878d2c5be0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:30.349643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c730549-5cb2-434b-a710-7f4d81048e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:30.350306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c730549-5cb2-434b-a710-7f4d81048e41 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:30.350590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c730549-5cb2-434b-a710-7f4d81048e41 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:30.350871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c730549-5cb2-434b-a710-7f4d81048e41 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:30.351517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c730549-5cb2-434b-a710-7f4d81048e41 None admin] RBAC: Adding query filter params (name=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:30.351822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c730549-5cb2-434b-a710-7f4d81048e41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:30.353247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c730549-5cb2-434b-a710-7f4d81048e41 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:30.358645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 368/718] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:48:30 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:30.364435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc1d8b-1f82-4a52-bccd-38ff455ac47c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:30.389840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:30.509997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcc1d8b-1f82-4a52-bccd-38ff455ac47c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:30.560350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1050196d-c360-47a2-9778-17153e773df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:30.560877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050196d-c360-47a2-9778-17153e773df6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:30.561153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050196d-c360-47a2-9778-17153e773df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:30.561419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1050196d-c360-47a2-9778-17153e773df6 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:30.562051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050196d-c360-47a2-9778-17153e773df6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:30.562373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050196d-c360-47a2-9778-17153e773df6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:30.563746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1050196d-c360-47a2-9778-17153e773df6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:30.573003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 351/719] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:48:30 2022] GET /identity/v3/services?type=network => generated 124 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:31.116059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ecbad-8813-4614-98d0-ed6f5c103043 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:31.116451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ecbad-8813-4614-98d0-ed6f5c103043 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:31.116631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ecbad-8813-4614-98d0-ed6f5c103043 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:31.118300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 369/720] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:31.123367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece20267-ab4a-40b8-a99f-25141fd6d9b1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:31.123865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece20267-ab4a-40b8-a99f-25141fd6d9b1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:31.123947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece20267-ab4a-40b8-a99f-25141fd6d9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:31.146613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ece20267-ab4a-40b8-a99f-25141fd6d9b1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:31.147812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ece20267-ab4a-40b8-a99f-25141fd6d9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:31.239623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 352/721] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:31.764793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ecbad-8813-4614-98d0-ed6f5c103043 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:31.900335 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ecbad-8813-4614-98d0-ed6f5c103043 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:31.947698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696a489e-8d30-4663-b3f7-d07c84dad27d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:31.948340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a489e-8d30-4663-b3f7-d07c84dad27d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:31.948620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a489e-8d30-4663-b3f7-d07c84dad27d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:31.948889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696a489e-8d30-4663-b3f7-d07c84dad27d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:31.963237 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696a489e-8d30-4663-b3f7-d07c84dad27d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-696a489e-8d30-4663-b3f7-d07c84dad27d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:31.968755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:31.969738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:31.971370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 370/722] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:31.975757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece20267-ab4a-40b8-a99f-25141fd6d9b1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:31.978992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:32.110546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece20267-ab4a-40b8-a99f-25141fd6d9b1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:32.122616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb077c-e685-4490-af04-aa3c02130088 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.122984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb077c-e685-4490-af04-aa3c02130088 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.123168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb077c-e685-4490-af04-aa3c02130088 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.124861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 371/723] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:32.139646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f590c1-41fc-48d1-b5cf-6116b8d5e8a2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.139929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f590c1-41fc-48d1-b5cf-6116b8d5e8a2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.140206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f590c1-41fc-48d1-b5cf-6116b8d5e8a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.152245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c466640-0aec-4213-af88-fb9a1308e888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:32.152878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c466640-0aec-4213-af88-fb9a1308e888 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.153070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c466640-0aec-4213-af88-fb9a1308e888 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.153344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c466640-0aec-4213-af88-fb9a1308e888 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.154023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c466640-0aec-4213-af88-fb9a1308e888 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:32.154358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c466640-0aec-4213-af88-fb9a1308e888 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:32.155751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c466640-0aec-4213-af88-fb9a1308e888 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:32.164559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 353/724] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:32.170295 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c466640-0aec-4213-af88-fb9a1308e888 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:32.179129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-33f590c1-41fc-48d1-b5cf-6116b8d5e8a2 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:32.180361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-33f590c1-41fc-48d1-b5cf-6116b8d5e8a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:32.214290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:32.286888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:32.323837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 372/725] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:48:32.325733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb0db-8aa9-4164-9187-471d5a2ce319 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.326025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb0db-8aa9-4164-9187-471d5a2ce319 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.326338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bbb0db-8aa9-4164-9187-471d5a2ce319 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.329784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c466640-0aec-4213-af88-fb9a1308e888 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:32.334470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 373/726] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:32.348929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf1e52-27fa-43e7-b598-8f658e29b09c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.349216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf1e52-27fa-43e7-b598-8f658e29b09c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.349495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcdf1e52-27fa-43e7-b598-8f658e29b09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.374471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef375dfd-0e09-4ca3-8d24-08a8e901ae6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:32.375413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef375dfd-0e09-4ca3-8d24-08a8e901ae6b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.375831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef375dfd-0e09-4ca3-8d24-08a8e901ae6b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.376273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef375dfd-0e09-4ca3-8d24-08a8e901ae6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.376616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dcdf1e52-27fa-43e7-b598-8f658e29b09c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:32.377962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dcdf1e52-27fa-43e7-b598-8f658e29b09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:32.390379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef375dfd-0e09-4ca3-8d24-08a8e901ae6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:32.392514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef375dfd-0e09-4ca3-8d24-08a8e901ae6b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ef375dfd-0e09-4ca3-8d24-08a8e901ae6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:32.412133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:32.413256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:32.414380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:32.414380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:32.414380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:32.416099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 354/727] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:32.421732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3edfca-5bff-43b9-903c-0d3434fcf45f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.421732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3edfca-5bff-43b9-903c-0d3434fcf45f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.421732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3edfca-5bff-43b9-903c-0d3434fcf45f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.425483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 355/728] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:32.427313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3edfca-5bff-43b9-903c-0d3434fcf45f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:32.469354 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 374/729] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 13 06:48:32.471111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219fd4-1dd9-44ee-8988-55fccfd47087 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.471617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219fd4-1dd9-44ee-8988-55fccfd47087 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.472081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0219fd4-1dd9-44ee-8988-55fccfd47087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.496132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a0219fd4-1dd9-44ee-8988-55fccfd47087 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:32.498667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a0219fd4-1dd9-44ee-8988-55fccfd47087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:32.547460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3edfca-5bff-43b9-903c-0d3434fcf45f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:32.592733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 375/730] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:32.593421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:32.622007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57a73f-3c06-4647-bb18-68b97233326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:32.622695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57a73f-3c06-4647-bb18-68b97233326b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.622974 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57a73f-3c06-4647-bb18-68b97233326b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.623245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57a73f-3c06-4647-bb18-68b97233326b None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.623895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57a73f-3c06-4647-bb18-68b97233326b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:32.624193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57a73f-3c06-4647-bb18-68b97233326b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:32.625567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57a73f-3c06-4647-bb18-68b97233326b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:32.642885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 356/731] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:32.648418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219fd4-1dd9-44ee-8988-55fccfd47087 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:32.676234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:32.712420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57a73f-3c06-4647-bb18-68b97233326b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:32.832338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57a73f-3c06-4647-bb18-68b97233326b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:32.841146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0219fd4-1dd9-44ee-8988-55fccfd47087 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:32.872579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675e5e3-13a5-4148-b2da-6cca4d967042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:32.873210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675e5e3-13a5-4148-b2da-6cca4d967042 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.873495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675e5e3-13a5-4148-b2da-6cca4d967042 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.873760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6675e5e3-13a5-4148-b2da-6cca4d967042 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.884488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6675e5e3-13a5-4148-b2da-6cca4d967042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6675e5e3-13a5-4148-b2da-6cca4d967042 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:32.889882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:32.890870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 357/732] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:32.892103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675e5e3-13a5-4148-b2da-6cca4d967042 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:32.894454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64d83a-9421-4d79-8fea-f99a95b65948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:32.895097 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64d83a-9421-4d79-8fea-f99a95b65948 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:32.895376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64d83a-9421-4d79-8fea-f99a95b65948 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:32.895646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b64d83a-9421-4d79-8fea-f99a95b65948 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:32.903366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64d83a-9421-4d79-8fea-f99a95b65948 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:32.904817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b64d83a-9421-4d79-8fea-f99a95b65948 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-7b64d83a-9421-4d79-8fea-f99a95b65948 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:32.916970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:32.918000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 376/733] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:32.919881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64d83a-9421-4d79-8fea-f99a95b65948 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:32.942568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:32.996933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:33.006767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6675e5e3-13a5-4148-b2da-6cca4d967042 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.030642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b64d83a-9421-4d79-8fea-f99a95b65948 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.043049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3859a-700e-4dcf-a1be-4c2398afaf0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.043765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3859a-700e-4dcf-a1be-4c2398afaf0b None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.044042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3859a-700e-4dcf-a1be-4c2398afaf0b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.044417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3859a-700e-4dcf-a1be-4c2398afaf0b None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.045073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3859a-700e-4dcf-a1be-4c2398afaf0b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:33.046527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3859a-700e-4dcf-a1be-4c2398afaf0b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:33.065809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfb63d-077b-4c48-84dc-a66e9a210453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.066566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 358/734] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:48:32 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:33.067435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3859a-700e-4dcf-a1be-4c2398afaf0b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:33.078519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfb63d-077b-4c48-84dc-a66e9a210453 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.078854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfb63d-077b-4c48-84dc-a66e9a210453 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.079138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cfb63d-077b-4c48-84dc-a66e9a210453 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.103281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cfb63d-077b-4c48-84dc-a66e9a210453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-47cfb63d-077b-4c48-84dc-a66e9a210453 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:33.111169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:33.112843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:33.114294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:33.117753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 377/735] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:33.118801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfb63d-077b-4c48-84dc-a66e9a210453 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:33.214574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:33.237969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3859a-700e-4dcf-a1be-4c2398afaf0b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.260147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cfb63d-077b-4c48-84dc-a66e9a210453 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.280230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56c205-1666-4b8c-92fb-007c86b2cb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.280855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56c205-1666-4b8c-92fb-007c86b2cb6c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.281133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56c205-1666-4b8c-92fb-007c86b2cb6c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.281411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb56c205-1666-4b8c-92fb-007c86b2cb6c None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.282154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56c205-1666-4b8c-92fb-007c86b2cb6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:33.282425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56c205-1666-4b8c-92fb-007c86b2cb6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:33.283816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb56c205-1666-4b8c-92fb-007c86b2cb6c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:33.295331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 359/736] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:33.296221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56c205-1666-4b8c-92fb-007c86b2cb6c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:33.299166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:33.307328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f73f05-b950-4d30-bee3-be0c80b8e437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.308306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f73f05-b950-4d30-bee3-be0c80b8e437 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.308701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f73f05-b950-4d30-bee3-be0c80b8e437 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.309600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f73f05-b950-4d30-bee3-be0c80b8e437 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.310382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f73f05-b950-4d30-bee3-be0c80b8e437 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:33.310697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f73f05-b950-4d30-bee3-be0c80b8e437 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:33.312727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f73f05-b950-4d30-bee3-be0c80b8e437 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:33.336328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 378/737] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Sep 13 06:48:33 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:33.337184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f73f05-b950-4d30-bee3-be0c80b8e437 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:33.415459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb56c205-1666-4b8c-92fb-007c86b2cb6c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.454960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6de6b2-5324-475b-88c4-0da2be587066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.455999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6de6b2-5324-475b-88c4-0da2be587066 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.456483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6de6b2-5324-475b-88c4-0da2be587066 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.456958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6de6b2-5324-475b-88c4-0da2be587066 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.457956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6de6b2-5324-475b-88c4-0da2be587066 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:33.458576 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6de6b2-5324-475b-88c4-0da2be587066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:33.460078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6de6b2-5324-475b-88c4-0da2be587066 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:33.469499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 360/738] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:33.470503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f73f05-b950-4d30-bee3-be0c80b8e437 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.474905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6de6b2-5324-475b-88c4-0da2be587066 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:33.496676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:33.532578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd626e18-892f-41fc-bc33-55f42ddd2cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.533534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd626e18-892f-41fc-bc33-55f42ddd2cee None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.533982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd626e18-892f-41fc-bc33-55f42ddd2cee None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.534476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd626e18-892f-41fc-bc33-55f42ddd2cee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.550420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd626e18-892f-41fc-bc33-55f42ddd2cee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:33.552733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd626e18-892f-41fc-bc33-55f42ddd2cee None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fd626e18-892f-41fc-bc33-55f42ddd2cee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:33.573392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:33.574776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:33.575855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:33.575855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:33.575855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:33.580128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 379/739] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:33.580845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd626e18-892f-41fc-bc33-55f42ddd2cee None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:33.587694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6de6b2-5324-475b-88c4-0da2be587066 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.626153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738a492-323a-452e-8001-70a37ced30bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.626768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738a492-323a-452e-8001-70a37ced30bb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.627044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738a492-323a-452e-8001-70a37ced30bb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.627310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8738a492-323a-452e-8001-70a37ced30bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.632074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:33.693321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd626e18-892f-41fc-bc33-55f42ddd2cee None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.700420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738a492-323a-452e-8001-70a37ced30bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:33.701894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8738a492-323a-452e-8001-70a37ced30bb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:33.721209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:33.730604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fc29d-d30b-4a60-90af-b98cab482792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.731532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc29d-d30b-4a60-90af-b98cab482792 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.731969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc29d-d30b-4a60-90af-b98cab482792 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.732379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fc29d-d30b-4a60-90af-b98cab482792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.743931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fc29d-d30b-4a60-90af-b98cab482792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:33.745709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fc29d-d30b-4a60-90af-b98cab482792 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c64fc29d-d30b-4a60-90af-b98cab482792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:33.761162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:33.762564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:33.763628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:33.763628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:33.763628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:33.765464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 380/740] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:33.766662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1354e-e9eb-4610-bc34-e0fe2d952ffe None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.766939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1354e-e9eb-4610-bc34-e0fe2d952ffe None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.767206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e1354e-e9eb-4610-bc34-e0fe2d952ffe None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.772491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 381/741] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:33.772491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1354e-e9eb-4610-bc34-e0fe2d952ffe None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:33.812486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 361/742] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:33 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 13 06:48:33.813634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738a492-323a-452e-8001-70a37ced30bb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:33.875801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:33.884628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e1354e-e9eb-4610-bc34-e0fe2d952ffe None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.917556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8738a492-323a-452e-8001-70a37ced30bb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:33.920493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64235fd-932d-419d-8632-d2b41900c41e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.921223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64235fd-932d-419d-8632-d2b41900c41e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.921509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64235fd-932d-419d-8632-d2b41900c41e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.921783 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64235fd-932d-419d-8632-d2b41900c41e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.922482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64235fd-932d-419d-8632-d2b41900c41e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:33.922791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64235fd-932d-419d-8632-d2b41900c41e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:33.924174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64235fd-932d-419d-8632-d2b41900c41e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:33.936739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 382/743] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:33.937091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:33.938073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ec02c-692d-4ff2-b89d-78f29b5e3657 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.938336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ec02c-692d-4ff2-b89d-78f29b5e3657 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.938671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67ec02c-692d-4ff2-b89d-78f29b5e3657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.957797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c67ec02c-692d-4ff2-b89d-78f29b5e3657 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:33.958156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549331d-b1fd-44ac-a613-9e5e2efcdbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:33.958774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549331d-b1fd-44ac-a613-9e5e2efcdbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:33.959011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c67ec02c-692d-4ff2-b89d-78f29b5e3657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:33.959112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549331d-b1fd-44ac-a613-9e5e2efcdbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:33.959315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9549331d-b1fd-44ac-a613-9e5e2efcdbf6 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:33.959959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549331d-b1fd-44ac-a613-9e5e2efcdbf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:33.960262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549331d-b1fd-44ac-a613-9e5e2efcdbf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:33.961652 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9549331d-b1fd-44ac-a613-9e5e2efcdbf6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:33.973986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 362/744] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:33.974656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549331d-b1fd-44ac-a613-9e5e2efcdbf6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.038182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 383/745] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:33 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 13 06:48:34.038182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ec02c-692d-4ff2-b89d-78f29b5e3657 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.097356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9549331d-b1fd-44ac-a613-9e5e2efcdbf6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.133447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:34.134401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be9d82-6e8c-4818-b6f0-679077a5f4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.135126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be9d82-6e8c-4818-b6f0-679077a5f4f3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.135490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be9d82-6e8c-4818-b6f0-679077a5f4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.135781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5be9d82-6e8c-4818-b6f0-679077a5f4f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:34.147962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be9d82-6e8c-4818-b6f0-679077a5f4f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:34.149428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5be9d82-6e8c-4818-b6f0-679077a5f4f3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:34.154539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67ec02c-692d-4ff2-b89d-78f29b5e3657 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d5be9d82-6e8c-4818-b6f0-679077a5f4f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:34.164840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:34.165773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:34.166707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:34.166707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:34.166707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:34.166707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 363/746] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:34.167198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be9d82-6e8c-4818-b6f0-679077a5f4f3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.203419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:34.206683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e803ba2-8738-4d07-9a21-e7ca4f607709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.207325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e803ba2-8738-4d07-9a21-e7ca4f607709 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.207600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e803ba2-8738-4d07-9a21-e7ca4f607709 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.207874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e803ba2-8738-4d07-9a21-e7ca4f607709 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:34.218745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e803ba2-8738-4d07-9a21-e7ca4f607709 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:34.220557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e803ba2-8738-4d07-9a21-e7ca4f607709 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2e803ba2-8738-4d07-9a21-e7ca4f607709 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:34.236688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:34.237739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:34.238711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 384/747] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Sep 13 06:48:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:34.240084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e803ba2-8738-4d07-9a21-e7ca4f607709 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.281025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5be9d82-6e8c-4818-b6f0-679077a5f4f3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.318281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7639-4e49-4a16-a2d0-7afad24f4f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.319099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7639-4e49-4a16-a2d0-7afad24f4f2d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.319477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7639-4e49-4a16-a2d0-7afad24f4f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.319854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f7639-4e49-4a16-a2d0-7afad24f4f2d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:34.329154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f7639-4e49-4a16-a2d0-7afad24f4f2d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:34.331343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4f7639-4e49-4a16-a2d0-7afad24f4f2d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-dc4f7639-4e49-4a16-a2d0-7afad24f4f2d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:34.345580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:34.346619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:34.347551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 364/748] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:48:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:34.348713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7639-4e49-4a16-a2d0-7afad24f4f2d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.361092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e803ba2-8738-4d07-9a21-e7ca4f607709 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.399639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3f029-28c0-4fac-b538-c534b5327d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.400255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3f029-28c0-4fac-b538-c534b5327d8e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.400530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3f029-28c0-4fac-b538-c534b5327d8e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.400799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca3f029-28c0-4fac-b538-c534b5327d8e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:34.401438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3f029-28c0-4fac-b538-c534b5327d8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:34.401734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3f029-28c0-4fac-b538-c534b5327d8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:34.403302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca3f029-28c0-4fac-b538-c534b5327d8e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:34.420903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 385/749] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:34.421794 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3f029-28c0-4fac-b538-c534b5327d8e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.425359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:34.444769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:34.456334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f7639-4e49-4a16-a2d0-7afad24f4f2d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.491709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c92d4-ca1d-4505-8e4a-c705924414e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.492256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c92d4-ca1d-4505-8e4a-c705924414e9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.492546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c92d4-ca1d-4505-8e4a-c705924414e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.492823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c92d4-ca1d-4505-8e4a-c705924414e9 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:34.493560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c92d4-ca1d-4505-8e4a-c705924414e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:34.493870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c92d4-ca1d-4505-8e4a-c705924414e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:34.495365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839c92d4-ca1d-4505-8e4a-c705924414e9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:34.513285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 365/750] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Sep 13 06:48:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:34.514120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c92d4-ca1d-4505-8e4a-c705924414e9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.530202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca3f029-28c0-4fac-b538-c534b5327d8e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.568318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b6875-8bf0-4308-b8cd-306a3673400a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.568933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b6875-8bf0-4308-b8cd-306a3673400a None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.569214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b6875-8bf0-4308-b8cd-306a3673400a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.569484 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f8b6875-8bf0-4308-b8cd-306a3673400a None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:34.570241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b6875-8bf0-4308-b8cd-306a3673400a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:34.571629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f8b6875-8bf0-4308-b8cd-306a3673400a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:34.588659 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 386/751] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:48:34 2022] POST /identity/v3/services => generated 240 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:34.589828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b6875-8bf0-4308-b8cd-306a3673400a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.632984 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c92d4-ca1d-4505-8e4a-c705924414e9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.673353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:34.678736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ddb08-7e81-4199-910c-716927209c91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.679378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ddb08-7e81-4199-910c-716927209c91 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.680786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ddb08-7e81-4199-910c-716927209c91 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.680786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8ddb08-7e81-4199-910c-716927209c91 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:34.681607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ddb08-7e81-4199-910c-716927209c91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:34.681964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ddb08-7e81-4199-910c-716927209c91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:34.684004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8ddb08-7e81-4199-910c-716927209c91 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:34.705200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 366/752] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Sep 13 06:48:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:34.705862 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ddb08-7e81-4199-910c-716927209c91 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.708818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:34.725203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f8b6875-8bf0-4308-b8cd-306a3673400a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.771647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc84165-e0ab-468a-b0a2-593249f9ad4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.772952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc84165-e0ab-468a-b0a2-593249f9ad4d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.773709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc84165-e0ab-468a-b0a2-593249f9ad4d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.774651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc84165-e0ab-468a-b0a2-593249f9ad4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:34.785127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc84165-e0ab-468a-b0a2-593249f9ad4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:34.788317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc84165-e0ab-468a-b0a2-593249f9ad4d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9bc84165-e0ab-468a-b0a2-593249f9ad4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:34.804917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:34.805875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:34.806822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:34.806822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:34.806822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:34.806822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:34.806822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:34.806822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:34.806822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:34.806822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:34.809954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 387/753] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:34.810648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc84165-e0ab-468a-b0a2-593249f9ad4d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:34.834971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8ddb08-7e81-4199-910c-716927209c91 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.928480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc84165-e0ab-468a-b0a2-593249f9ad4d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:34.955479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:34.970446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e21386-83b9-4a12-ae01-3b5e6188fb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.971799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e21386-83b9-4a12-ae01-3b5e6188fb9f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.972478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e21386-83b9-4a12-ae01-3b5e6188fb9f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.972894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e21386-83b9-4a12-ae01-3b5e6188fb9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:34.997425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981a53b-cabd-4bf4-a94d-97179305c740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:34.998461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981a53b-cabd-4bf4-a94d-97179305c740 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:34.998924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981a53b-cabd-4bf4-a94d-97179305c740 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:34.999334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8981a53b-cabd-4bf4-a94d-97179305c740 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.011140 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981a53b-cabd-4bf4-a94d-97179305c740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:35.013010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8981a53b-cabd-4bf4-a94d-97179305c740 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-8981a53b-cabd-4bf4-a94d-97179305c740 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:35.030099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:35.031240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:35.032343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:35.032343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:35.032343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:35.038222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 367/754] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:35.039365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981a53b-cabd-4bf4-a94d-97179305c740 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:35.057624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e21386-83b9-4a12-ae01-3b5e6188fb9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:35.060201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e21386-83b9-4a12-ae01-3b5e6188fb9f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:35.103779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:35.155096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8981a53b-cabd-4bf4-a94d-97179305c740 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:35.177484 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 388/755] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:34 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:48:35.178757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e21386-83b9-4a12-ae01-3b5e6188fb9f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:35.195752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94368c5-6b4a-4dd6-b625-e93b5dc5f609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:35.197192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94368c5-6b4a-4dd6-b625-e93b5dc5f609 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:35.197618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94368c5-6b4a-4dd6-b625-e93b5dc5f609 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:35.198305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94368c5-6b4a-4dd6-b625-e93b5dc5f609 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.199839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94368c5-6b4a-4dd6-b625-e93b5dc5f609 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:35.200407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94368c5-6b4a-4dd6-b625-e93b5dc5f609 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:35.202257 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94368c5-6b4a-4dd6-b625-e93b5dc5f609 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:35.209345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:35.224357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 368/756] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:35.225629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94368c5-6b4a-4dd6-b625-e93b5dc5f609 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:35.294696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e21386-83b9-4a12-ae01-3b5e6188fb9f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:35.333777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b4403-f46e-4486-9231-6fd475f83212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:35.334584 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b4403-f46e-4486-9231-6fd475f83212 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:35.334981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b4403-f46e-4486-9231-6fd475f83212 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:35.335358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6b4403-f46e-4486-9231-6fd475f83212 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.336010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b4403-f46e-4486-9231-6fd475f83212 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:35.336318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b4403-f46e-4486-9231-6fd475f83212 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:35.337735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f6b4403-f46e-4486-9231-6fd475f83212 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:35.350385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 389/757] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:35.350666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b4403-f46e-4486-9231-6fd475f83212 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:35.353314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94368c5-6b4a-4dd6-b625-e93b5dc5f609 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:35.368269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:35.391226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c77e95-fc28-4ddf-be07-eee409028d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:35.391874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77e95-fc28-4ddf-be07-eee409028d0a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:35.392149 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77e95-fc28-4ddf-be07-eee409028d0a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:35.392419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c77e95-fc28-4ddf-be07-eee409028d0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.404094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c77e95-fc28-4ddf-be07-eee409028d0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:35.405684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c77e95-fc28-4ddf-be07-eee409028d0a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-98c77e95-fc28-4ddf-be07-eee409028d0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:35.423264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:35.424433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:35.425350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:35.425350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:35.425350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:35.425350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 369/758] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:35.425708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee805d7-c874-47e8-80cd-c61dcce338f7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:35.425925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee805d7-c874-47e8-80cd-c61dcce338f7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:35.426515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee805d7-c874-47e8-80cd-c61dcce338f7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.427770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 370/759] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:35.428809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee805d7-c874-47e8-80cd-c61dcce338f7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:35.447453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:35.464319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6b4403-f46e-4486-9231-6fd475f83212 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:35.498723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5864f14-b8cb-42b3-94ac-22d82eda78d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:35.499282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5864f14-b8cb-42b3-94ac-22d82eda78d8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:35.499699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5864f14-b8cb-42b3-94ac-22d82eda78d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:35.499960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5864f14-b8cb-42b3-94ac-22d82eda78d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.500748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5864f14-b8cb-42b3-94ac-22d82eda78d8 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=4815ef133a3a4604bc467c222d450fbf) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:35.501119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5864f14-b8cb-42b3-94ac-22d82eda78d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:35.502669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5864f14-b8cb-42b3-94ac-22d82eda78d8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:35.508330 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 390/760] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:35 2022] GET /identity/v3/role_assignments?user.id=4815ef133a3a4604bc467c222d450fbf&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:35.520402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e2e1e-3cce-4aa3-a63e-ed46ead3aa7d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:35.521365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e2e1e-3cce-4aa3-a63e-ed46ead3aa7d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:35.522099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e2e1e-3cce-4aa3-a63e-ed46ead3aa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.547181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee805d7-c874-47e8-80cd-c61dcce338f7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:35.549378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6a0e2e1e-3cce-4aa3-a63e-ed46ead3aa7d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:35.551321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6a0e2e1e-3cce-4aa3-a63e-ed46ead3aa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:35.605044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba76e1-1bee-4342-a8c9-4363adc6d48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:35.606000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba76e1-1bee-4342-a8c9-4363adc6d48f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:35.606474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba76e1-1bee-4342-a8c9-4363adc6d48f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:35.606884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ba76e1-1bee-4342-a8c9-4363adc6d48f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.617979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba76e1-1bee-4342-a8c9-4363adc6d48f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:35.620411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ba76e1-1bee-4342-a8c9-4363adc6d48f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b0ba76e1-1bee-4342-a8c9-4363adc6d48f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:35.638898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:35.640023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:35.641111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:35.641111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:35.641111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:35.641111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:35.641111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:35.641111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:35.641111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:35.641111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:35.647971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 391/761] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:35.647971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e2e1e-3cce-4aa3-a63e-ed46ead3aa7d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:35.648542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 371/762] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:35.650741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba76e1-1bee-4342-a8c9-4363adc6d48f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:35.711646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:35.787413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:35.792659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ba76e1-1bee-4342-a8c9-4363adc6d48f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:35.831885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13289c3b-d083-4e22-8c73-277c460314f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:35.832416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13289c3b-d083-4e22-8c73-277c460314f5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:35.832690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13289c3b-d083-4e22-8c73-277c460314f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:35.832956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13289c3b-d083-4e22-8c73-277c460314f5 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.833642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13289c3b-d083-4e22-8c73-277c460314f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:35.833941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13289c3b-d083-4e22-8c73-277c460314f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:35.839609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13289c3b-d083-4e22-8c73-277c460314f5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:35.853744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e2e1e-3cce-4aa3-a63e-ed46ead3aa7d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:35.866970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 372/763] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:35.872643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13289c3b-d083-4e22-8c73-277c460314f5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:35.891485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a4d93-d9dd-4f27-a1a5-50f0c4677e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:35.892104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a4d93-d9dd-4f27-a1a5-50f0c4677e05 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:35.892379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a4d93-d9dd-4f27-a1a5-50f0c4677e05 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:35.892646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a4d93-d9dd-4f27-a1a5-50f0c4677e05 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:35.893284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a4d93-d9dd-4f27-a1a5-50f0c4677e05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:35.893580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a4d93-d9dd-4f27-a1a5-50f0c4677e05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:35.894977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3a4d93-d9dd-4f27-a1a5-50f0c4677e05 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:35.908833 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 392/764] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:35.919198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a4d93-d9dd-4f27-a1a5-50f0c4677e05 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:35.983083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:35.998696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13289c3b-d083-4e22-8c73-277c460314f5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:36.033929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec91b0-2496-4d6e-827d-d7d6c3438d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:36.034636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec91b0-2496-4d6e-827d-d7d6c3438d75 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:36.034915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec91b0-2496-4d6e-827d-d7d6c3438d75 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:36.035193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aec91b0-2496-4d6e-827d-d7d6c3438d75 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/87f3a9ffb88f4faa85edc3dabce0c8f5/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:36.037133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec91b0-2496-4d6e-827d-d7d6c3438d75 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=87f3a9ffb88f4faa85edc3dabce0c8f5, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:36.038643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aec91b0-2496-4d6e-827d-d7d6c3438d75 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:36.043282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a4d93-d9dd-4f27-a1a5-50f0c4677e05 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:36.056072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 373/765] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:48:35 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/87f3a9ffb88f4faa85edc3dabce0c8f5/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:48:36.057437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec91b0-2496-4d6e-827d-d7d6c3438d75 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:36.080656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b75e1-2734-4d99-b76c-cb50c5909615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:36.081316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b75e1-2734-4d99-b76c-cb50c5909615 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:36.081594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b75e1-2734-4d99-b76c-cb50c5909615 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:36.081865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905b75e1-2734-4d99-b76c-cb50c5909615 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:36.089849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b75e1-2734-4d99-b76c-cb50c5909615 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:36.091404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905b75e1-2734-4d99-b76c-cb50c5909615 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:36.094721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-905b75e1-2734-4d99-b76c-cb50c5909615 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:36.106537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:36.107524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:36.108824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:36.108824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:36.108824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:36.108824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:36.108824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:36.108824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:36.108824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:36.108824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:36.108824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 393/766] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:36.119199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b75e1-2734-4d99-b76c-cb50c5909615 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:36.179839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aec91b0-2496-4d6e-827d-d7d6c3438d75 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:36.218041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb033b3-15d3-4a95-8a1b-fe360766d4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:36.218780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb033b3-15d3-4a95-8a1b-fe360766d4e3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:36.219060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb033b3-15d3-4a95-8a1b-fe360766d4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:36.219380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb033b3-15d3-4a95-8a1b-fe360766d4e3 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:36.219982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb033b3-15d3-4a95-8a1b-fe360766d4e3 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:36.221375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb033b3-15d3-4a95-8a1b-fe360766d4e3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6bb033b3-15d3-4a95-8a1b-fe360766d4e3 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:36.231051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:36.232096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:36.234101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 374/767] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Sep 13 06:48:36 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:36.239829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb033b3-15d3-4a95-8a1b-fe360766d4e3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:36.254145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:36.356765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb033b3-15d3-4a95-8a1b-fe360766d4e3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:36.379652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905b75e1-2734-4d99-b76c-cb50c5909615 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:36.412176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8ef4c-725e-4188-80c5-8f16060a404a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:36.412799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8ef4c-725e-4188-80c5-8f16060a404a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:36.413072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8ef4c-725e-4188-80c5-8f16060a404a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:36.413340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8ef4c-725e-4188-80c5-8f16060a404a None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:36.413975 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8ef4c-725e-4188-80c5-8f16060a404a None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:36.414310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8ef4c-725e-4188-80c5-8f16060a404a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:36.415689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8ef4c-725e-4188-80c5-8f16060a404a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:36.427121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 375/768] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Sep 13 06:48:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:36.432200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8ef4c-725e-4188-80c5-8f16060a404a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:36.432499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abb9b1-b519-419a-96c1-2d65899107c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:36.434012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abb9b1-b519-419a-96c1-2d65899107c1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:36.434689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abb9b1-b519-419a-96c1-2d65899107c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:36.435164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8abb9b1-b519-419a-96c1-2d65899107c1 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:36.436632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abb9b1-b519-419a-96c1-2d65899107c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:36.437205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abb9b1-b519-419a-96c1-2d65899107c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:36.438995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8abb9b1-b519-419a-96c1-2d65899107c1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:36.454063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 394/769] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:36.466854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abb9b1-b519-419a-96c1-2d65899107c1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:36.527582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:36.543354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8ef4c-725e-4188-80c5-8f16060a404a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:36.598964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:36.617670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b57c9d-f2e3-40c2-95ab-bcb521a79146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:36.618387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b57c9d-f2e3-40c2-95ab-bcb521a79146 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:36.618660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b57c9d-f2e3-40c2-95ab-bcb521a79146 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:36.618926 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b57c9d-f2e3-40c2-95ab-bcb521a79146 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:36.619556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b57c9d-f2e3-40c2-95ab-bcb521a79146 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:36.619854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b57c9d-f2e3-40c2-95ab-bcb521a79146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:36.622563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b57c9d-f2e3-40c2-95ab-bcb521a79146 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:36.634477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 376/770] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Sep 13 06:48:36 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:36.640043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b57c9d-f2e3-40c2-95ab-bcb521a79146 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:36.656279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8abb9b1-b519-419a-96c1-2d65899107c1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:36.714969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5cc2c-880e-414d-a91a-58c36027edf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:36.715606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5cc2c-880e-414d-a91a-58c36027edf0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:36.715888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5cc2c-880e-414d-a91a-58c36027edf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:36.716164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c5cc2c-880e-414d-a91a-58c36027edf0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:36.716908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5cc2c-880e-414d-a91a-58c36027edf0 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=7363637f44264a65b66ba53da825eaff) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:36.717218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5cc2c-880e-414d-a91a-58c36027edf0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:36.718736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c5cc2c-880e-414d-a91a-58c36027edf0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:36.734864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 395/771] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:36 2022] GET /identity/v3/role_assignments?user.id=7363637f44264a65b66ba53da825eaff&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 259 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:36.785293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b57c9d-f2e3-40c2-95ab-bcb521a79146 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:36.825430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0097869-78b2-41cb-a6ed-51b8a2d44451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:36.826514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0097869-78b2-41cb-a6ed-51b8a2d44451 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:36.826983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0097869-78b2-41cb-a6ed-51b8a2d44451 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:36.827441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0097869-78b2-41cb-a6ed-51b8a2d44451 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:36.828344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0097869-78b2-41cb-a6ed-51b8a2d44451 None admin] RBAC: Adding query filter params (interface=public, service_id=ab74d89f19b14a5e84eec2a4ff11a029, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:36.828865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0097869-78b2-41cb-a6ed-51b8a2d44451 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:36.830477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0097869-78b2-41cb-a6ed-51b8a2d44451 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:36.839698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 377/772] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Sep 13 06:48:36 2022] GET /identity/v3/endpoints?service_id=ab74d89f19b14a5e84eec2a4ff11a029&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:36.845751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5cc2c-880e-414d-a91a-58c36027edf0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:36.972207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c5cc2c-880e-414d-a91a-58c36027edf0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:36.996830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:37.008986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a860e18-2ebb-49cd-8689-ac8708a8df16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:37.009905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a860e18-2ebb-49cd-8689-ac8708a8df16 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:37.010393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a860e18-2ebb-49cd-8689-ac8708a8df16 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:37.010868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a860e18-2ebb-49cd-8689-ac8708a8df16 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:37.011899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a860e18-2ebb-49cd-8689-ac8708a8df16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:37.013701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a860e18-2ebb-49cd-8689-ac8708a8df16 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:37.019603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 396/773] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Sep 13 06:48:36 2022] GET /identity/v3/services => generated 772 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:37.165169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b9bf3-4e16-41f2-b293-aa6a3d00fbc1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:37.165818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b9bf3-4e16-41f2-b293-aa6a3d00fbc1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:37.166358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9b9bf3-4e16-41f2-b293-aa6a3d00fbc1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:37.168850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 378/774] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:37.174419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ca2c3-18a7-4d46-9291-58e0f81dc387 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:37.175374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ca2c3-18a7-4d46-9291-58e0f81dc387 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:37.175840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3ca2c3-18a7-4d46-9291-58e0f81dc387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:37.201138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5b3ca2c3-18a7-4d46-9291-58e0f81dc387 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:37.202754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5b3ca2c3-18a7-4d46-9291-58e0f81dc387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:37.296885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 397/775] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:37.573871 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b9bf3-4e16-41f2-b293-aa6a3d00fbc1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:37.577197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:37.723791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9b9bf3-4e16-41f2-b293-aa6a3d00fbc1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:37.772732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5727df-e841-49a8-a441-ba263b446a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:37.773384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5727df-e841-49a8-a441-ba263b446a02 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:37.773666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5727df-e841-49a8-a441-ba263b446a02 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:37.773944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5727df-e841-49a8-a441-ba263b446a02 None admin] PATH_INFO: `/v3/services/network` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:37.774572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5727df-e841-49a8-a441-ba263b446a02 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:37.775963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5727df-e841-49a8-a441-ba263b446a02 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2f5727df-e841-49a8-a441-ba263b446a02 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:37.791334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:37.792478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:37.793470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 379/776] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:37 2022] GET /identity/v3/services/network => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:37.798790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ca2c3-18a7-4d46-9291-58e0f81dc387 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:37.960422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3ca2c3-18a7-4d46-9291-58e0f81dc387 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:37.969807 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fa015-b389-4f42-86dc-652bb0ef5108 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:37.971543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fa015-b389-4f42-86dc-652bb0ef5108 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:37.971543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fa015-b389-4f42-86dc-652bb0ef5108 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:37.971985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 380/777] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:37.977121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd36e75-b416-4d83-be80-9545fa097b77 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:37.977508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd36e75-b416-4d83-be80-9545fa097b77 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:37.977877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd36e75-b416-4d83-be80-9545fa097b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:37.982441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:38.017473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cbd36e75-b416-4d83-be80-9545fa097b77 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:38.022953 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cbd36e75-b416-4d83-be80-9545fa097b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:38.024087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bce0cf-38c3-400a-9ef2-280e3f7d185e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:38.025337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bce0cf-38c3-400a-9ef2-280e3f7d185e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:38.026112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bce0cf-38c3-400a-9ef2-280e3f7d185e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:38.026788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bce0cf-38c3-400a-9ef2-280e3f7d185e None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:38.028033 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bce0cf-38c3-400a-9ef2-280e3f7d185e None admin] RBAC: Adding query filter params (name=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:38.031270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bce0cf-38c3-400a-9ef2-280e3f7d185e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:38.033322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bce0cf-38c3-400a-9ef2-280e3f7d185e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:38.050730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 398/778] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:48:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:38.055946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bce0cf-38c3-400a-9ef2-280e3f7d185e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:38.140421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:38.146658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 381/779] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:48:38.219649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bce0cf-38c3-400a-9ef2-280e3f7d185e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:38.261469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64fb01-15c6-4e3b-814a-030840e28fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:38.266296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64fb01-15c6-4e3b-814a-030840e28fba None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:38.266600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64fb01-15c6-4e3b-814a-030840e28fba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:38.266881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b64fb01-15c6-4e3b-814a-030840e28fba None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:38.267586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64fb01-15c6-4e3b-814a-030840e28fba None admin] RBAC: Adding query filter params (type=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:38.267937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64fb01-15c6-4e3b-814a-030840e28fba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:38.269433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b64fb01-15c6-4e3b-814a-030840e28fba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:38.274850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 399/780] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:48:38 2022] GET /identity/v3/services?type=network => generated 350 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:38.280582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd36e75-b416-4d83-be80-9545fa097b77 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:38.410937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd36e75-b416-4d83-be80-9545fa097b77 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:38.430767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64fb01-15c6-4e3b-814a-030840e28fba None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:38.459728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e9106-efea-4879-82b7-d17f282c066b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:38.460354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e9106-efea-4879-82b7-d17f282c066b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:38.460633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e9106-efea-4879-82b7-d17f282c066b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:38.460895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e9106-efea-4879-82b7-d17f282c066b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:38.461599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e9106-efea-4879-82b7-d17f282c066b None admin] RBAC: Adding query filter params (interface=public, service_id=cd1bc33eed7243ff88ba0f42d8f0c7de, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:38.461903 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e9106-efea-4879-82b7-d17f282c066b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:38.463311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e9106-efea-4879-82b7-d17f282c066b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:38.478072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 382/781] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Sep 13 06:48:38 2022] GET /identity/v3/endpoints?service_id=cd1bc33eed7243ff88ba0f42d8f0c7de&interface=public®ion_id=RegionOne => generated 194 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:38.483107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e9106-efea-4879-82b7-d17f282c066b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:38.506874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:38.588904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b64fb01-15c6-4e3b-814a-030840e28fba None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:38.597781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:38.663106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7a2ee-e74f-4e4a-8e77-d5128362ef7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:38.663768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7a2ee-e74f-4e4a-8e77-d5128362ef7f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:38.664058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7a2ee-e74f-4e4a-8e77-d5128362ef7f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:38.664328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de7a2ee-e74f-4e4a-8e77-d5128362ef7f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:38.681087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de7a2ee-e74f-4e4a-8e77-d5128362ef7f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:38.684731 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e9106-efea-4879-82b7-d17f282c066b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6de7a2ee-e74f-4e4a-8e77-d5128362ef7f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:38.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:38.687666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:38.688661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 400/782] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:38.694106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7a2ee-e74f-4e4a-8e77-d5128362ef7f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:38.760571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329eeca-6dc9-41a1-ad0c-796db0583119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:38.761069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329eeca-6dc9-41a1-ad0c-796db0583119 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:38.761345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329eeca-6dc9-41a1-ad0c-796db0583119 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:38.761664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329eeca-6dc9-41a1-ad0c-796db0583119 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:38.762463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329eeca-6dc9-41a1-ad0c-796db0583119 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:38.763860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329eeca-6dc9-41a1-ad0c-796db0583119 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:38.775427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 383/783] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Sep 13 06:48:38 2022] GET /identity/v3/services => generated 772 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:38.784338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f590e-ff4d-4929-b29d-aa3578ac5fdd None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:38.784682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f590e-ff4d-4929-b29d-aa3578ac5fdd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:38.784898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355f590e-ff4d-4929-b29d-aa3578ac5fdd None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:38.786517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 384/784] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:38.797254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3c659-1b4f-4d8c-af1e-e4b8a1309e4c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:38.797691 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3c659-1b4f-4d8c-af1e-e4b8a1309e4c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:38.798105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3c659-1b4f-4d8c-af1e-e4b8a1309e4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:38.819734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-36e3c659-1b4f-4d8c-af1e-e4b8a1309e4c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:38.821084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-36e3c659-1b4f-4d8c-af1e-e4b8a1309e4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:38.900368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 385/785] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:48:38.906059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de7a2ee-e74f-4e4a-8e77-d5128362ef7f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:38.916585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:38.968931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88359440-bec3-445d-a7a9-5c82879b0ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:38.969571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88359440-bec3-445d-a7a9-5c82879b0ae5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:38.969897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88359440-bec3-445d-a7a9-5c82879b0ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:38.970200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88359440-bec3-445d-a7a9-5c82879b0ae5 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:38.970884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88359440-bec3-445d-a7a9-5c82879b0ae5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:38.971193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88359440-bec3-445d-a7a9-5c82879b0ae5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:38.972628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88359440-bec3-445d-a7a9-5c82879b0ae5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:38.987186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 401/786] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:38.993103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3c659-1b4f-4d8c-af1e-e4b8a1309e4c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:39.053676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:39.100078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3c659-1b4f-4d8c-af1e-e4b8a1309e4c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:39.134447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa023-1013-468c-91ac-7c0d4710e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:39.135089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa023-1013-468c-91ac-7c0d4710e3f7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.135423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa023-1013-468c-91ac-7c0d4710e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.135699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcfa023-1013-468c-91ac-7c0d4710e3f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.146338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfa023-1013-468c-91ac-7c0d4710e3f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:39.147821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcfa023-1013-468c-91ac-7c0d4710e3f7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-cfcfa023-1013-468c-91ac-7c0d4710e3f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:39.162929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:39.163873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:39.165168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:39.165168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:39.165168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:39.165168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 386/787] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:39.170229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88359440-bec3-445d-a7a9-5c82879b0ae5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:39.265650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa023-1013-468c-91ac-7c0d4710e3f7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:39.282923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88359440-bec3-445d-a7a9-5c82879b0ae5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:39.350114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f83ccb-663c-4984-8e19-700eed759484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:39.350968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f83ccb-663c-4984-8e19-700eed759484 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.351463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f83ccb-663c-4984-8e19-700eed759484 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.351973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f83ccb-663c-4984-8e19-700eed759484 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.352696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f83ccb-663c-4984-8e19-700eed759484 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:39.353010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f83ccb-663c-4984-8e19-700eed759484 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:39.354488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f83ccb-663c-4984-8e19-700eed759484 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:39.381677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 402/788] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:39.387501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f83ccb-663c-4984-8e19-700eed759484 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:39.437678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:39.446676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcfa023-1013-468c-91ac-7c0d4710e3f7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:39.464419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:39.512260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852e37d-571e-4cae-820d-1fa6b2ef44c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:39.513451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852e37d-571e-4cae-820d-1fa6b2ef44c8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.513733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852e37d-571e-4cae-820d-1fa6b2ef44c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.514009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852e37d-571e-4cae-820d-1fa6b2ef44c8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.528351 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852e37d-571e-4cae-820d-1fa6b2ef44c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9852e37d-571e-4cae-820d-1fa6b2ef44c8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:39.534032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:39.534992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:39.535841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:39.538367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 387/789] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:39.539931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c652f-c56b-4f8d-86ca-b1ff956b4a74 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.540420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c652f-c56b-4f8d-86ca-b1ff956b4a74 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.540962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c652f-c56b-4f8d-86ca-b1ff956b4a74 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.542539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 388/790] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:39.543274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c652f-c56b-4f8d-86ca-b1ff956b4a74 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:39.564564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f83ccb-663c-4984-8e19-700eed759484 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:39.615420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d124bfc-86e3-464e-9336-8458357e480b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:39.616553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d124bfc-86e3-464e-9336-8458357e480b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.617400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d124bfc-86e3-464e-9336-8458357e480b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.617859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d124bfc-86e3-464e-9336-8458357e480b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.627315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d124bfc-86e3-464e-9336-8458357e480b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:39.630317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d124bfc-86e3-464e-9336-8458357e480b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1d124bfc-86e3-464e-9336-8458357e480b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:39.646725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:39.647920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:39.649108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:39.652011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 403/791] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:39.653537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ac5d6-106f-4c80-9580-1db918097132 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.653979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ac5d6-106f-4c80-9580-1db918097132 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.654401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ac5d6-106f-4c80-9580-1db918097132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.669403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c652f-c56b-4f8d-86ca-b1ff956b4a74 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:39.677802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-391ac5d6-106f-4c80-9580-1db918097132 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:39.679764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-391ac5d6-106f-4c80-9580-1db918097132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:39.707701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bfe44-7277-47b2-afc6-a142951e8b31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:39.708451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bfe44-7277-47b2-afc6-a142951e8b31 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.708840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bfe44-7277-47b2-afc6-a142951e8b31 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.709204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7bfe44-7277-47b2-afc6-a142951e8b31 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.710008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bfe44-7277-47b2-afc6-a142951e8b31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:39.710426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bfe44-7277-47b2-afc6-a142951e8b31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:39.711939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c7bfe44-7277-47b2-afc6-a142951e8b31 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:39.725885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 389/792] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:39.725885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bfe44-7277-47b2-afc6-a142951e8b31 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:39.754914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:39.778167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 404/793] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:39.779370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ac5d6-106f-4c80-9580-1db918097132 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:39.799997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:39.896586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ac5d6-106f-4c80-9580-1db918097132 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:39.921080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7bfe44-7277-47b2-afc6-a142951e8b31 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:39.934441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f4b6c-e93a-4a17-9d90-4da90e14c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:39.935479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f4b6c-e93a-4a17-9d90-4da90e14c9fc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.936007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f4b6c-e93a-4a17-9d90-4da90e14c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.936477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f4b6c-e93a-4a17-9d90-4da90e14c9fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.948021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f4b6c-e93a-4a17-9d90-4da90e14c9fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:39.950235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f4b6c-e93a-4a17-9d90-4da90e14c9fc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:39.956753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8776ea2d-b5a4-487f-9c90-0c577cbd2c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:39.957720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ea2d-b5a4-487f-9c90-0c577cbd2c83 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.957720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ea2d-b5a4-487f-9c90-0c577cbd2c83 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.957976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8776ea2d-b5a4-487f-9c90-0c577cbd2c83 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.958744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776ea2d-b5a4-487f-9c90-0c577cbd2c83 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:39.959053 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776ea2d-b5a4-487f-9c90-0c577cbd2c83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:39.960788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8776ea2d-b5a4-487f-9c90-0c577cbd2c83 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-eb4f4b6c-e93a-4a17-9d90-4da90e14c9fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:39.967473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:39.968446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:39.969363 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:39.969363 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:39.969363 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:39.971294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 405/794] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:39.972730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e49c4-bd70-4011-b91b-b2bfbb2ab2a4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.973217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e49c4-bd70-4011-b91b-b2bfbb2ab2a4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.973706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694e49c4-bd70-4011-b91b-b2bfbb2ab2a4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:39.975567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 406/795] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:39.977489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e49c4-bd70-4011-b91b-b2bfbb2ab2a4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:39.986264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 390/796] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Sep 13 06:48:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:39.987595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc8e3a-84a1-4490-89a0-7d609ec6b013 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:39.987923 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc8e3a-84a1-4490-89a0-7d609ec6b013 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:39.988198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cc8e3a-84a1-4490-89a0-7d609ec6b013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.007973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-37cc8e3a-84a1-4490-89a0-7d609ec6b013 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:40.009265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-37cc8e3a-84a1-4490-89a0-7d609ec6b013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:40.065318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:40.087299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 391/797] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:40.088356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc8e3a-84a1-4490-89a0-7d609ec6b013 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.109945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:40.122384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694e49c4-bd70-4011-b91b-b2bfbb2ab2a4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.172392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2057d71-bb58-4e83-8fa7-1de3f2f8a47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.172934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2057d71-bb58-4e83-8fa7-1de3f2f8a47e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.173229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2057d71-bb58-4e83-8fa7-1de3f2f8a47e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.173488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2057d71-bb58-4e83-8fa7-1de3f2f8a47e None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.174170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2057d71-bb58-4e83-8fa7-1de3f2f8a47e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:40.174475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2057d71-bb58-4e83-8fa7-1de3f2f8a47e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:40.175852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2057d71-bb58-4e83-8fa7-1de3f2f8a47e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:40.195845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 407/798] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:40.195845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2057d71-bb58-4e83-8fa7-1de3f2f8a47e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.211702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cc8e3a-84a1-4490-89a0-7d609ec6b013 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.248570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c066c-e0cd-4f24-a413-ce14da5170d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.249569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c066c-e0cd-4f24-a413-ce14da5170d3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.250037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c066c-e0cd-4f24-a413-ce14da5170d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.250565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17c066c-e0cd-4f24-a413-ce14da5170d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.261371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c066c-e0cd-4f24-a413-ce14da5170d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:40.263088 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17c066c-e0cd-4f24-a413-ce14da5170d3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c17c066c-e0cd-4f24-a413-ce14da5170d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:40.278760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:40.279746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:40.280640 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:40.280640 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:40.280640 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:40.282712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 392/799] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:40.283726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c066c-e0cd-4f24-a413-ce14da5170d3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.321951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2057d71-bb58-4e83-8fa7-1de3f2f8a47e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.352392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:40.361112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6cc42-2646-4952-9d3f-d2f461032866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.361754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6cc42-2646-4952-9d3f-d2f461032866 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.362029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6cc42-2646-4952-9d3f-d2f461032866 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.362335 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6cc42-2646-4952-9d3f-d2f461032866 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.362830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:40.373832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6cc42-2646-4952-9d3f-d2f461032866 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9ae6cc42-2646-4952-9d3f-d2f461032866 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:40.379732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:40.380664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:40.381503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 408/800] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:40.382222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6cc42-2646-4952-9d3f-d2f461032866 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.405545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17c066c-e0cd-4f24-a413-ce14da5170d3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.445401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8603d-4665-401b-ad94-e78f3dfc022e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.445947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8603d-4665-401b-ad94-e78f3dfc022e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.446254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8603d-4665-401b-ad94-e78f3dfc022e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.446528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8603d-4665-401b-ad94-e78f3dfc022e None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.454814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8603d-4665-401b-ad94-e78f3dfc022e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:40.456362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f8603d-4665-401b-ad94-e78f3dfc022e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-74f8603d-4665-401b-ad94-e78f3dfc022e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:40.471516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:40.472486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 393/801] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:40.473452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8603d-4665-401b-ad94-e78f3dfc022e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.491950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6cc42-2646-4952-9d3f-d2f461032866 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.526509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece7432-e802-4f50-a96e-e8b3f525b19b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.527154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece7432-e802-4f50-a96e-e8b3f525b19b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.527431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece7432-e802-4f50-a96e-e8b3f525b19b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.527700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ece7432-e802-4f50-a96e-e8b3f525b19b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.528336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece7432-e802-4f50-a96e-e8b3f525b19b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:40.528637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece7432-e802-4f50-a96e-e8b3f525b19b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:40.530005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ece7432-e802-4f50-a96e-e8b3f525b19b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:40.541463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 409/802] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:40.543040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece7432-e802-4f50-a96e-e8b3f525b19b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.579043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8603d-4665-401b-ad94-e78f3dfc022e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.593430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:40.612700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:40.623906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e341b-1d82-4df0-9741-0847af8e8be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.624544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e341b-1d82-4df0-9741-0847af8e8be0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.624819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e341b-1d82-4df0-9741-0847af8e8be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.625497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590e341b-1d82-4df0-9741-0847af8e8be0 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.626102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e341b-1d82-4df0-9741-0847af8e8be0 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:40.627861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590e341b-1d82-4df0-9741-0847af8e8be0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-590e341b-1d82-4df0-9741-0847af8e8be0 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:40.637849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:40.638879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:40.639736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 394/803] 10.222.0.25 () {56 vars in 1129 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:40.640552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e341b-1d82-4df0-9741-0847af8e8be0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.655807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ece7432-e802-4f50-a96e-e8b3f525b19b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.689728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5b3f0-dba4-4106-af69-1c77f31cdf54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.690806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5b3f0-dba4-4106-af69-1c77f31cdf54 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.690806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5b3f0-dba4-4106-af69-1c77f31cdf54 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.691157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a5b3f0-dba4-4106-af69-1c77f31cdf54 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.691913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5b3f0-dba4-4106-af69-1c77f31cdf54 None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:40.692271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5b3f0-dba4-4106-af69-1c77f31cdf54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:40.693755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a5b3f0-dba4-4106-af69-1c77f31cdf54 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:40.702889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 410/804] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:40.704000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5b3f0-dba4-4106-af69-1c77f31cdf54 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.746935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590e341b-1d82-4df0-9741-0847af8e8be0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.783400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48d03c-5632-4969-9d35-34a668669358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.784005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48d03c-5632-4969-9d35-34a668669358 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.784263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48d03c-5632-4969-9d35-34a668669358 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.784557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f48d03c-5632-4969-9d35-34a668669358 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.785259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48d03c-5632-4969-9d35-34a668669358 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:40.785560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48d03c-5632-4969-9d35-34a668669358 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:40.787026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f48d03c-5632-4969-9d35-34a668669358 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:40.805278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 395/805] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:40.806132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48d03c-5632-4969-9d35-34a668669358 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.815805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a5b3f0-dba4-4106-af69-1c77f31cdf54 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.827712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:40.853188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cedba0-c7f5-4d72-abcd-ae1df4a59859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.854115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cedba0-c7f5-4d72-abcd-ae1df4a59859 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.854542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cedba0-c7f5-4d72-abcd-ae1df4a59859 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.854958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cedba0-c7f5-4d72-abcd-ae1df4a59859 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.860186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:40.862681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cedba0-c7f5-4d72-abcd-ae1df4a59859 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:40.864146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9cedba0-c7f5-4d72-abcd-ae1df4a59859 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-a9cedba0-c7f5-4d72-abcd-ae1df4a59859 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:40.875676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:40.876613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 411/806] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:40.882449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cedba0-c7f5-4d72-abcd-ae1df4a59859 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.919689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f48d03c-5632-4969-9d35-34a668669358 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:40.955496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e13ac2-782a-444b-81b0-6d3e05d278c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:40.956027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e13ac2-782a-444b-81b0-6d3e05d278c0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:40.956299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e13ac2-782a-444b-81b0-6d3e05d278c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:40.956565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e13ac2-782a-444b-81b0-6d3e05d278c0 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:40.957193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e13ac2-782a-444b-81b0-6d3e05d278c0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:40.957492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e13ac2-782a-444b-81b0-6d3e05d278c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:40.958897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e13ac2-782a-444b-81b0-6d3e05d278c0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:40.963676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 396/807] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:40.965005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e13ac2-782a-444b-81b0-6d3e05d278c0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:40.989218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cedba0-c7f5-4d72-abcd-ae1df4a59859 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:41.024243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b450-51bb-43fd-806f-1a569ff675fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:41.024878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b450-51bb-43fd-806f-1a569ff675fe None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.025174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b450-51bb-43fd-806f-1a569ff675fe None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.025425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f4b450-51bb-43fd-806f-1a569ff675fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.036365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4b450-51bb-43fd-806f-1a569ff675fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:41.037811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f4b450-51bb-43fd-806f-1a569ff675fe None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-43f4b450-51bb-43fd-806f-1a569ff675fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:41.052307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:41.053309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:41.054229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:41.054229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:41.054229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:41.054229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 412/808] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:41.054229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b450-51bb-43fd-806f-1a569ff675fe None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:41.081379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e13ac2-782a-444b-81b0-6d3e05d278c0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:41.085221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:41.097066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:41.125455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31886437-9dc2-4b8a-a5fb-d86d5f15b8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:41.126514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31886437-9dc2-4b8a-a5fb-d86d5f15b8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.126995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31886437-9dc2-4b8a-a5fb-d86d5f15b8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.127555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31886437-9dc2-4b8a-a5fb-d86d5f15b8c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.165964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f4b450-51bb-43fd-806f-1a569ff675fe None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:41.202413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31886437-9dc2-4b8a-a5fb-d86d5f15b8c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:41.203901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31886437-9dc2-4b8a-a5fb-d86d5f15b8c6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:41.204790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b3435-a62c-4796-b63d-0364672840d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:41.205424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b3435-a62c-4796-b63d-0364672840d7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.205753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b3435-a62c-4796-b63d-0364672840d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.205970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630b3435-a62c-4796-b63d-0364672840d7 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.206712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b3435-a62c-4796-b63d-0364672840d7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:41.207016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b3435-a62c-4796-b63d-0364672840d7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:41.208485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630b3435-a62c-4796-b63d-0364672840d7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:41.222070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 413/809] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:41.222955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b3435-a62c-4796-b63d-0364672840d7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:41.305730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:41.322931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 397/810] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:40 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:48:41.322931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31886437-9dc2-4b8a-a5fb-d86d5f15b8c6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:41.324960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630b3435-a62c-4796-b63d-0364672840d7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:41.334207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:41.358679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c40f5-6577-4318-9979-42f20d25a9ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:41.359292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c40f5-6577-4318-9979-42f20d25a9ad None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.359566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c40f5-6577-4318-9979-42f20d25a9ad None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.359835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486c40f5-6577-4318-9979-42f20d25a9ad None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.360477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c40f5-6577-4318-9979-42f20d25a9ad None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:41.360776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c40f5-6577-4318-9979-42f20d25a9ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:41.362151 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-486c40f5-6577-4318-9979-42f20d25a9ad None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:41.371685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 414/811] 10.222.0.25 () {56 vars in 1124 bytes} [Tue Sep 13 06:48:41 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:41.371685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c40f5-6577-4318-9979-42f20d25a9ad None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:41.423739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31886437-9dc2-4b8a-a5fb-d86d5f15b8c6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:41.458610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f05a7f-4560-48ba-b44a-cf410808affa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:41.459174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f05a7f-4560-48ba-b44a-cf410808affa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.459454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f05a7f-4560-48ba-b44a-cf410808affa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.459725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f05a7f-4560-48ba-b44a-cf410808affa None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.460366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f05a7f-4560-48ba-b44a-cf410808affa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:41.460668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f05a7f-4560-48ba-b44a-cf410808affa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:41.462062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f05a7f-4560-48ba-b44a-cf410808affa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:41.471163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486c40f5-6577-4318-9979-42f20d25a9ad None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:41.478047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 398/812] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:41.478047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2054d2b-bdee-42c7-b974-b8270e261ae9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.478047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2054d2b-bdee-42c7-b974-b8270e261ae9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.478047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2054d2b-bdee-42c7-b974-b8270e261ae9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.478047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 399/813] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:41.478047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2054d2b-bdee-42c7-b974-b8270e261ae9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:41.506895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed95da-8aab-49c9-9f8f-c947b1edc9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:41.507563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed95da-8aab-49c9-9f8f-c947b1edc9f2 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.507841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed95da-8aab-49c9-9f8f-c947b1edc9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.508113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ed95da-8aab-49c9-9f8f-c947b1edc9f2 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/4815ef133a3a4604bc467c222d450fbf/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.510005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed95da-8aab-49c9-9f8f-c947b1edc9f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=4815ef133a3a4604bc467c222d450fbf, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:41.511944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ed95da-8aab-49c9-9f8f-c947b1edc9f2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:41.525313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 415/814] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:48:41 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/4815ef133a3a4604bc467c222d450fbf/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:48:41.526093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed95da-8aab-49c9-9f8f-c947b1edc9f2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:41.536034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:41.568166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:41.603883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2054d2b-bdee-42c7-b974-b8270e261ae9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:41.649586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235e14b-445a-4100-b565-e8169422d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:41.650288 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235e14b-445a-4100-b565-e8169422d130 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.650565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235e14b-445a-4100-b565-e8169422d130 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.650834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5235e14b-445a-4100-b565-e8169422d130 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.663302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235e14b-445a-4100-b565-e8169422d130 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:41.665172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5235e14b-445a-4100-b565-e8169422d130 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5235e14b-445a-4100-b565-e8169422d130 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:41.683773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:41.684817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:41.690819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:41.690819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:41.690819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:41.690819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 400/815] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:41.690819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235e14b-445a-4100-b565-e8169422d130 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:41.776168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ed95da-8aab-49c9-9f8f-c947b1edc9f2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:41.799720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5235e14b-445a-4100-b565-e8169422d130 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:41.815772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0af745-264c-4a78-8f51-0551bb27498e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:41.816386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0af745-264c-4a78-8f51-0551bb27498e None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.816668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0af745-264c-4a78-8f51-0551bb27498e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.816937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0af745-264c-4a78-8f51-0551bb27498e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.817610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0af745-264c-4a78-8f51-0551bb27498e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:41.819042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0af745-264c-4a78-8f51-0551bb27498e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:41.826543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:41.836584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df790d7c-905c-4240-a4b8-0df3dbc317eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:41.837335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df790d7c-905c-4240-a4b8-0df3dbc317eb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.837622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df790d7c-905c-4240-a4b8-0df3dbc317eb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.837944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df790d7c-905c-4240-a4b8-0df3dbc317eb None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.846863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df790d7c-905c-4240-a4b8-0df3dbc317eb None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:41.848309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df790d7c-905c-4240-a4b8-0df3dbc317eb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:41.854464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 416/816] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:48:41 2022] POST /identity/v3/endpoints => generated 346 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:41.856523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e4dea-8de9-4879-9905-e19162580be6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:41.856815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e4dea-8de9-4879-9905-e19162580be6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:41.857089 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388e4dea-8de9-4879-9905-e19162580be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-df790d7c-905c-4240-a4b8-0df3dbc317eb None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:41.860916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:41.861916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:41.863549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:41.866275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 401/817] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:48:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:41.867765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df790d7c-905c-4240-a4b8-0df3dbc317eb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:41.892013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-388e4dea-8de9-4879-9905-e19162580be6 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:41.893982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-388e4dea-8de9-4879-9905-e19162580be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:41.964411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:41.985756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 417/818] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:41.986930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e4dea-8de9-4879-9905-e19162580be6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.046817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df790d7c-905c-4240-a4b8-0df3dbc317eb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.095570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0d59d-c62b-45e1-b65d-42f5de3e4563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.096197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0d59d-c62b-45e1-b65d-42f5de3e4563 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.096474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0d59d-c62b-45e1-b65d-42f5de3e4563 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.096743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e0d59d-c62b-45e1-b65d-42f5de3e4563 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.097391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0d59d-c62b-45e1-b65d-42f5de3e4563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:42.097696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0d59d-c62b-45e1-b65d-42f5de3e4563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.099141 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e0d59d-c62b-45e1-b65d-42f5de3e4563 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.120189 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 402/819] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:42.125587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0d59d-c62b-45e1-b65d-42f5de3e4563 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.131101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388e4dea-8de9-4879-9905-e19162580be6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.173967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47ac0-c87b-4789-aee2-aeafcfc00898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.174614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47ac0-c87b-4789-aee2-aeafcfc00898 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.174894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47ac0-c87b-4789-aee2-aeafcfc00898 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.175171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b47ac0-c87b-4789-aee2-aeafcfc00898 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.175998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b47ac0-c87b-4789-aee2-aeafcfc00898 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:42.177500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:42.177633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b47ac0-c87b-4789-aee2-aeafcfc00898 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.179450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b47ac0-c87b-4789-aee2-aeafcfc00898 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.200943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 418/820] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Sep 13 06:48:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:42.208685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47ac0-c87b-4789-aee2-aeafcfc00898 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.249777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e0d59d-c62b-45e1-b65d-42f5de3e4563 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.277948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:42.318183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062857d-7525-4f5d-b413-7801ea0539b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.319084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062857d-7525-4f5d-b413-7801ea0539b6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.319611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062857d-7525-4f5d-b413-7801ea0539b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.319880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6062857d-7525-4f5d-b413-7801ea0539b6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.328605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062857d-7525-4f5d-b413-7801ea0539b6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.330062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6062857d-7525-4f5d-b413-7801ea0539b6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6062857d-7525-4f5d-b413-7801ea0539b6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:42.342506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:42.343607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 403/821] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:42.344707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062857d-7525-4f5d-b413-7801ea0539b6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.356671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b47ac0-c87b-4789-aee2-aeafcfc00898 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.404380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b255f11-eed4-4249-81d5-7afab9716fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.405650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b255f11-eed4-4249-81d5-7afab9716fb5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.406007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b255f11-eed4-4249-81d5-7afab9716fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.406611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b255f11-eed4-4249-81d5-7afab9716fb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.421420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b255f11-eed4-4249-81d5-7afab9716fb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.424079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b255f11-eed4-4249-81d5-7afab9716fb5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-0b255f11-eed4-4249-81d5-7afab9716fb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:42.444113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:42.445123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:42.446032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:42.446032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:42.446032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:42.446032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 419/822] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:42.446032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b255f11-eed4-4249-81d5-7afab9716fb5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.459919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6062857d-7525-4f5d-b413-7801ea0539b6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.467551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:42.498327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7aae88-d0ee-470c-aa0b-80175a82d130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.498971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7aae88-d0ee-470c-aa0b-80175a82d130 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.499247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7aae88-d0ee-470c-aa0b-80175a82d130 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.499512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7aae88-d0ee-470c-aa0b-80175a82d130 None admin] PATH_INFO: `/v3/services/network` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.500302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7aae88-d0ee-470c-aa0b-80175a82d130 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.501467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7aae88-d0ee-470c-aa0b-80175a82d130 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-cf7aae88-d0ee-470c-aa0b-80175a82d130 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:42.510614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:42.512033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 13 06:48:42.512907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:42.516665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 404/823] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:42 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:42.517492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7aae88-d0ee-470c-aa0b-80175a82d130 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.568337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:42.572779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b255f11-eed4-4249-81d5-7afab9716fb5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.621573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb98a9e-73cc-402f-8a21-15fe32b3ef7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.622474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb98a9e-73cc-402f-8a21-15fe32b3ef7a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.622857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb98a9e-73cc-402f-8a21-15fe32b3ef7a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.623224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb98a9e-73cc-402f-8a21-15fe32b3ef7a None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.624127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb98a9e-73cc-402f-8a21-15fe32b3ef7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:42.624523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb98a9e-73cc-402f-8a21-15fe32b3ef7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.626052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb98a9e-73cc-402f-8a21-15fe32b3ef7a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.626794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7aae88-d0ee-470c-aa0b-80175a82d130 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.643501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 420/824] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:42.644643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb98a9e-73cc-402f-8a21-15fe32b3ef7a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.665348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d6a82-4992-4212-9d48-361a45c49efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.666216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d6a82-4992-4212-9d48-361a45c49efd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.666633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d6a82-4992-4212-9d48-361a45c49efd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.667040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d6a82-4992-4212-9d48-361a45c49efd None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.667994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d6a82-4992-4212-9d48-361a45c49efd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:42.668482 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d6a82-4992-4212-9d48-361a45c49efd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.670190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d6a82-4992-4212-9d48-361a45c49efd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.684998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 405/825] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:42.686175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d6a82-4992-4212-9d48-361a45c49efd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.721154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:42.757863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb98a9e-73cc-402f-8a21-15fe32b3ef7a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.792898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d8fa5-cf08-49a1-a939-fedc76f58454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.793874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d8fa5-cf08-49a1-a939-fedc76f58454 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.794192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d8fa5-cf08-49a1-a939-fedc76f58454 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.794460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6d8fa5-cf08-49a1-a939-fedc76f58454 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.795086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d8fa5-cf08-49a1-a939-fedc76f58454 None admin] RBAC: Adding query filter params (name=network) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:42.795383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d8fa5-cf08-49a1-a939-fedc76f58454 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.796852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6d8fa5-cf08-49a1-a939-fedc76f58454 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.802832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 421/826] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:48:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:42.803636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d8fa5-cf08-49a1-a939-fedc76f58454 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.809377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d6a82-4992-4212-9d48-361a45c49efd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.830324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:42.849768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17411-0c29-4bcb-82c5-4118b23cb0e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.850848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17411-0c29-4bcb-82c5-4118b23cb0e7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.851371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17411-0c29-4bcb-82c5-4118b23cb0e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.851906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc17411-0c29-4bcb-82c5-4118b23cb0e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.852956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17411-0c29-4bcb-82c5-4118b23cb0e7 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=87f3a9ffb88f4faa85edc3dabce0c8f5) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:42.853461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17411-0c29-4bcb-82c5-4118b23cb0e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.855326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc17411-0c29-4bcb-82c5-4118b23cb0e7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.870513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 406/827] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:42 2022] GET /identity/v3/role_assignments?user.id=87f3a9ffb88f4faa85edc3dabce0c8f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:42.872146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17411-0c29-4bcb-82c5-4118b23cb0e7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:42.925558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6d8fa5-cf08-49a1-a939-fedc76f58454 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:42.968392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d80fa8-54cc-4495-872f-ceef4cc4de63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:42.969044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d80fa8-54cc-4495-872f-ceef4cc4de63 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:42.969321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d80fa8-54cc-4495-872f-ceef4cc4de63 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:42.969593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d80fa8-54cc-4495-872f-ceef4cc4de63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:42.978770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d80fa8-54cc-4495-872f-ceef4cc4de63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:42.981459 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d80fa8-54cc-4495-872f-ceef4cc4de63 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-53d80fa8-54cc-4495-872f-ceef4cc4de63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:42.996774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:42.997764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:42.998692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:42.998692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:42.998692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:42.998692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:42.998692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:42.998692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:42.998692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:42.998692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:42.998692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 422/828] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:43.005722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d80fa8-54cc-4495-872f-ceef4cc4de63 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:43.050316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:43.063483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc17411-0c29-4bcb-82c5-4118b23cb0e7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:43.130696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412a77f-9829-40f8-ab41-2d9ef7900525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:43.131641 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412a77f-9829-40f8-ab41-2d9ef7900525 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:43.132721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412a77f-9829-40f8-ab41-2d9ef7900525 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:43.133300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e412a77f-9829-40f8-ab41-2d9ef7900525 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:43.134048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412a77f-9829-40f8-ab41-2d9ef7900525 None admin] RBAC: Adding query filter params (type=network) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:43.138853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412a77f-9829-40f8-ab41-2d9ef7900525 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:43.141419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e412a77f-9829-40f8-ab41-2d9ef7900525 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:43.151416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 407/829] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:48:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:43.155550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:43.158018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412a77f-9829-40f8-ab41-2d9ef7900525 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:43.193786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d80fa8-54cc-4495-872f-ceef4cc4de63 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:43.233657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ae55a-665e-4fb0-99d9-2ac7fccfb993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:43.234816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ae55a-665e-4fb0-99d9-2ac7fccfb993 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:43.235312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ae55a-665e-4fb0-99d9-2ac7fccfb993 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:43.235781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ae55a-665e-4fb0-99d9-2ac7fccfb993 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:43.236671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ae55a-665e-4fb0-99d9-2ac7fccfb993 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:43.237233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ae55a-665e-4fb0-99d9-2ac7fccfb993 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:43.239013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ae55a-665e-4fb0-99d9-2ac7fccfb993 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:43.253536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 423/830] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:43.259866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ae55a-665e-4fb0-99d9-2ac7fccfb993 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:43.280186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e412a77f-9829-40f8-ab41-2d9ef7900525 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:43.318703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da50143-5f02-4b7f-acb8-199a82efaef9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:43.319747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50143-5f02-4b7f-acb8-199a82efaef9 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:43.319747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50143-5f02-4b7f-acb8-199a82efaef9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:43.320022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da50143-5f02-4b7f-acb8-199a82efaef9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:43.320822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da50143-5f02-4b7f-acb8-199a82efaef9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:43.322262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da50143-5f02-4b7f-acb8-199a82efaef9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:43.349063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 408/831] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:48:43 2022] POST /identity/v3/endpoints => generated 337 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:43.383883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ae55a-665e-4fb0-99d9-2ac7fccfb993 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:43.441874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ca542-8bd7-4a30-a9a4-c8287b5dd762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:43.442848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ca542-8bd7-4a30-a9a4-c8287b5dd762 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:43.443276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ca542-8bd7-4a30-a9a4-c8287b5dd762 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:43.443690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ca542-8bd7-4a30-a9a4-c8287b5dd762 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/7363637f44264a65b66ba53da825eaff/roles/df32bbf3791f4028b2dabf497d5e938d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:43.454139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:43.464425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ca542-8bd7-4a30-a9a4-c8287b5dd762 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=7363637f44264a65b66ba53da825eaff, role_id=df32bbf3791f4028b2dabf497d5e938d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:43.466747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ca542-8bd7-4a30-a9a4-c8287b5dd762 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:43.485889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 424/832] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:48:43 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/7363637f44264a65b66ba53da825eaff/roles/df32bbf3791f4028b2dabf497d5e938d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 06:48:43.848035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504b82d-1df7-4f55-96f2-b217cd1a604a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:43.848571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504b82d-1df7-4f55-96f2-b217cd1a604a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:43.848571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5504b82d-1df7-4f55-96f2-b217cd1a604a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:43.850229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 409/833] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:43.856078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa65ab-a1d8-48ab-9393-c7ddc9a4c09a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:43.856356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa65ab-a1d8-48ab-9393-c7ddc9a4c09a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:43.856631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51aa65ab-a1d8-48ab-9393-c7ddc9a4c09a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:43.875028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-51aa65ab-a1d8-48ab-9393-c7ddc9a4c09a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:43.876244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-51aa65ab-a1d8-48ab-9393-c7ddc9a4c09a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:43.954186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 425/834] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:43 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:44.213410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504b82d-1df7-4f55-96f2-b217cd1a604a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:44.229877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:44.327032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5504b82d-1df7-4f55-96f2-b217cd1a604a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:44.369242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc245a2-608a-49ed-a342-e67f0c043819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:44.369883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc245a2-608a-49ed-a342-e67f0c043819 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:44.370285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc245a2-608a-49ed-a342-e67f0c043819 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:44.370564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc245a2-608a-49ed-a342-e67f0c043819 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:44.381063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc245a2-608a-49ed-a342-e67f0c043819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6fc245a2-608a-49ed-a342-e67f0c043819 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:44.386151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:44.387125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:48:44.387944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:44.388594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909ea8-26e8-45dd-8412-834629ca2a9d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:44.388594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909ea8-26e8-45dd-8412-834629ca2a9d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:44.388594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c909ea8-26e8-45dd-8412-834629ca2a9d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:44.389671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 426/836] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:44.391901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 410/836] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:44.393967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c909ea8-26e8-45dd-8412-834629ca2a9d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:44.395632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec350369-31f9-42a6-8f3e-31e70b06ea33 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:44.395916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec350369-31f9-42a6-8f3e-31e70b06ea33 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:44.396191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec350369-31f9-42a6-8f3e-31e70b06ea33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:44.414729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ec350369-31f9-42a6-8f3e-31e70b06ea33 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:44.415985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ec350369-31f9-42a6-8f3e-31e70b06ea33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:44.494030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 411/837] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:44 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:44.509081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c909ea8-26e8-45dd-8412-834629ca2a9d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:44.530265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:44.547939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7aa28-90db-4a03-a734-1006b2d20d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:44.548560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7aa28-90db-4a03-a734-1006b2d20d81 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:44.548848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7aa28-90db-4a03-a734-1006b2d20d81 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:44.549118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f7aa28-90db-4a03-a734-1006b2d20d81 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:44.549784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7aa28-90db-4a03-a734-1006b2d20d81 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:44.550103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7aa28-90db-4a03-a734-1006b2d20d81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:44.551689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f7aa28-90db-4a03-a734-1006b2d20d81 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:44.560984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 427/838] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:48:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:44.566770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec350369-31f9-42a6-8f3e-31e70b06ea33 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:44.571530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:44.673993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec350369-31f9-42a6-8f3e-31e70b06ea33 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:44.709979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040ed52-9f03-4be5-ac5c-541332ca5ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:44.710908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040ed52-9f03-4be5-ac5c-541332ca5ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:44.711336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040ed52-9f03-4be5-ac5c-541332ca5ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:44.711750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d040ed52-9f03-4be5-ac5c-541332ca5ce7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:44.722525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040ed52-9f03-4be5-ac5c-541332ca5ce7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:44.724178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d040ed52-9f03-4be5-ac5c-541332ca5ce7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-d040ed52-9f03-4be5-ac5c-541332ca5ce7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:44.740095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:44.741282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:44.742458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:44.742458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:44.742458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:44.748335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 412/839] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:44.750587 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7aa28-90db-4a03-a734-1006b2d20d81 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:44.777682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040ed52-9f03-4be5-ac5c-541332ca5ce7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:44.849937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:44.875891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f7aa28-90db-4a03-a734-1006b2d20d81 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:44.909514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d040ed52-9f03-4be5-ac5c-541332ca5ce7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:44.913028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b516b29-d3ae-41e7-a577-b290f54749d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:44.913738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b516b29-d3ae-41e7-a577-b290f54749d7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:44.914138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b516b29-d3ae-41e7-a577-b290f54749d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:44.914507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b516b29-d3ae-41e7-a577-b290f54749d7 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:44.915350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b516b29-d3ae-41e7-a577-b290f54749d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:44.915757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b516b29-d3ae-41e7-a577-b290f54749d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:44.917211 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b516b29-d3ae-41e7-a577-b290f54749d7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:44.934130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 428/840] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:44.938933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b516b29-d3ae-41e7-a577-b290f54749d7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:44.953150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fb990-bfd5-48b6-9ea7-e5f9a0a56cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:44.953788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fb990-bfd5-48b6-9ea7-e5f9a0a56cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:44.954063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fb990-bfd5-48b6-9ea7-e5f9a0a56cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:44.954386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04fb990-bfd5-48b6-9ea7-e5f9a0a56cf6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:44.955047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fb990-bfd5-48b6-9ea7-e5f9a0a56cf6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:44.956476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04fb990-bfd5-48b6-9ea7-e5f9a0a56cf6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e04fb990-bfd5-48b6-9ea7-e5f9a0a56cf6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:44.966264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:44.967258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:44.968452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 413/841] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:44 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:44.973128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fb990-bfd5-48b6-9ea7-e5f9a0a56cf6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:44.993294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:45.066130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b516b29-d3ae-41e7-a577-b290f54749d7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:45.100293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe369c-8b8a-4cc0-8b86-f6bf21ea73ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:45.101234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe369c-8b8a-4cc0-8b86-f6bf21ea73ac None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.101604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe369c-8b8a-4cc0-8b86-f6bf21ea73ac None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.101967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbe369c-8b8a-4cc0-8b86-f6bf21ea73ac None admin] PATH_INFO: `/v3/users/swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.105969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04fb990-bfd5-48b6-9ea7-e5f9a0a56cf6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:45.111084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe369c-8b8a-4cc0-8b86-f6bf21ea73ac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:45.112632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbe369c-8b8a-4cc0-8b86-f6bf21ea73ac None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-fbbe369c-8b8a-4cc0-8b86-f6bf21ea73ac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:45.127191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:45.128297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 13 06:48:45.129247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:45.132098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:45.134159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 429/842] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:48:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:45.135246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe369c-8b8a-4cc0-8b86-f6bf21ea73ac None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:45.151252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ee717-075b-4f2a-962c-255f765a669b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:45.152267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ee717-075b-4f2a-962c-255f765a669b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.152951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ee717-075b-4f2a-962c-255f765a669b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.153447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9ee717-075b-4f2a-962c-255f765a669b None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.154290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ee717-075b-4f2a-962c-255f765a669b None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:45.154787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ee717-075b-4f2a-962c-255f765a669b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:45.156345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9ee717-075b-4f2a-962c-255f765a669b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:45.161497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 414/843] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:44 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:45.169146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ee717-075b-4f2a-962c-255f765a669b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:45.244255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:45.247563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbe369c-8b8a-4cc0-8b86-f6bf21ea73ac None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:45.290867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97ae0c-8c1c-494e-ae12-02026e844a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:45.291789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97ae0c-8c1c-494e-ae12-02026e844a0e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.292236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97ae0c-8c1c-494e-ae12-02026e844a0e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.292362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9ee717-075b-4f2a-962c-255f765a669b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:45.293186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97ae0c-8c1c-494e-ae12-02026e844a0e None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.294397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97ae0c-8c1c-494e-ae12-02026e844a0e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:45.294848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97ae0c-8c1c-494e-ae12-02026e844a0e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:45.297438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e97ae0c-8c1c-494e-ae12-02026e844a0e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:45.321608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 430/844] 10.222.0.25 () {56 vars in 1135 bytes} [Tue Sep 13 06:48:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:45.328710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97ae0c-8c1c-494e-ae12-02026e844a0e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:45.337302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8499861d-a5e4-4cd0-84e0-5aa08fe5eb16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:45.338008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499861d-a5e4-4cd0-84e0-5aa08fe5eb16 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.338394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499861d-a5e4-4cd0-84e0-5aa08fe5eb16 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.338742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8499861d-a5e4-4cd0-84e0-5aa08fe5eb16 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.339503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499861d-a5e4-4cd0-84e0-5aa08fe5eb16 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:45.339878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499861d-a5e4-4cd0-84e0-5aa08fe5eb16 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:45.341835 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8499861d-a5e4-4cd0-84e0-5aa08fe5eb16 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:45.348337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 415/845] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:45 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:45.453596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97ae0c-8c1c-494e-ae12-02026e844a0e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:45.492801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cf9b9-3ccb-4c35-a3dd-a052ae8dbaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:45.493712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cf9b9-3ccb-4c35-a3dd-a052ae8dbaf0 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.494157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cf9b9-3ccb-4c35-a3dd-a052ae8dbaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.494590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5cf9b9-3ccb-4c35-a3dd-a052ae8dbaf0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.525304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:45.582891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cf9b9-3ccb-4c35-a3dd-a052ae8dbaf0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:45.584318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5cf9b9-3ccb-4c35-a3dd-a052ae8dbaf0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:45.680710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ebdc8-5151-4bfd-8607-c5e464eb8d26 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.681014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ebdc8-5151-4bfd-8607-c5e464eb8d26 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.681258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ebdc8-5151-4bfd-8607-c5e464eb8d26 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.683499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 416/846] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:45.690908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 431/847] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:45 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:48:45.691154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ed9be-e60e-4467-a9de-1d7190628d7a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.691384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ed9be-e60e-4467-a9de-1d7190628d7a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.692559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3ed9be-e60e-4467-a9de-1d7190628d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.695835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cf9b9-3ccb-4c35-a3dd-a052ae8dbaf0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:45.711205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fd3ed9be-e60e-4467-a9de-1d7190628d7a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:45.712520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fd3ed9be-e60e-4467-a9de-1d7190628d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:45.755178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:45.762073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:45.798183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 417/848] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:45.806801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5cf9b9-3ccb-4c35-a3dd-a052ae8dbaf0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:45.842054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d1c025-076d-43e5-bbf7-4df6edd45d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:45.842732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1c025-076d-43e5-bbf7-4df6edd45d8c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.843011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1c025-076d-43e5-bbf7-4df6edd45d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.843286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d1c025-076d-43e5-bbf7-4df6edd45d8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.853811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1c025-076d-43e5-bbf7-4df6edd45d8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:45.855396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d1c025-076d-43e5-bbf7-4df6edd45d8c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-42d1c025-076d-43e5-bbf7-4df6edd45d8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:45.870544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:45.871811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:45.872807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:45.872807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:45.872807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:45.876250 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 432/849] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:45.878202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ed9be-e60e-4467-a9de-1d7190628d7a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:45.896996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab437d-114d-4d2a-812b-060c5c7b7400 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.897336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab437d-114d-4d2a-812b-060c5c7b7400 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.897543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ab437d-114d-4d2a-812b-060c5c7b7400 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.899154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 433/850] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:45.904267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6c212-1361-4100-83c6-bac48024e8dc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:45.904547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6c212-1361-4100-83c6-bac48024e8dc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:45.904826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b6c212-1361-4100-83c6-bac48024e8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:45.923641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-52b6c212-1361-4100-83c6-bac48024e8dc None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:45.924911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-52b6c212-1361-4100-83c6-bac48024e8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:45.996604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3ed9be-e60e-4467-a9de-1d7190628d7a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:45.997732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 434/851] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:45 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:46.032035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fe36c-4d7f-4c22-bc95-6861ba763957 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.032740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe36c-4d7f-4c22-bc95-6861ba763957 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.033024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe36c-4d7f-4c22-bc95-6861ba763957 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.033297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89fe36c-4d7f-4c22-bc95-6861ba763957 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.033944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fe36c-4d7f-4c22-bc95-6861ba763957 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:46.034279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fe36c-4d7f-4c22-bc95-6861ba763957 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:46.035662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89fe36c-4d7f-4c22-bc95-6861ba763957 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:46.051760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 418/852] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:46.053138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6c212-1361-4100-83c6-bac48024e8dc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:46.071830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fe36c-4d7f-4c22-bc95-6861ba763957 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:46.107005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:46.114411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:46.186325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b6c212-1361-4100-83c6-bac48024e8dc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:46.191491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89fe36c-4d7f-4c22-bc95-6861ba763957 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:46.229004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba52608-612c-483c-af9c-07d97f5bea55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.229397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f5506-5bfb-4cf7-afe6-2d4104cb3069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.230070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f5506-5bfb-4cf7-afe6-2d4104cb3069 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.230376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f5506-5bfb-4cf7-afe6-2d4104cb3069 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.230648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752f5506-5bfb-4cf7-afe6-2d4104cb3069 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.231245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba52608-612c-483c-af9c-07d97f5bea55 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.231793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba52608-612c-483c-af9c-07d97f5bea55 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.232260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba52608-612c-483c-af9c-07d97f5bea55 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.233032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba52608-612c-483c-af9c-07d97f5bea55 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:46.234641 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba52608-612c-483c-af9c-07d97f5bea55 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:46.238409 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f5506-5bfb-4cf7-afe6-2d4104cb3069 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:46.240013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752f5506-5bfb-4cf7-afe6-2d4104cb3069 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-eba52608-612c-483c-af9c-07d97f5bea55 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:46.244673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:46.245883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 06:48:46.247290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-752f5506-5bfb-4cf7-afe6-2d4104cb3069 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:46.254157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:46.255057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:46.255962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:46.255962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:46.255962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:46.255962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:46.255962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:46.255962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:46.255962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:46.255962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:46.257095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 419/854] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:46.257214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 435/854] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:46.258786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f5506-5bfb-4cf7-afe6-2d4104cb3069 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:46.259346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba52608-612c-483c-af9c-07d97f5bea55 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:46.367580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba52608-612c-483c-af9c-07d97f5bea55 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:46.368886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:46.370313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:46.372002 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752f5506-5bfb-4cf7-afe6-2d4104cb3069 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:46.405074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a16a56-8a60-4983-87cf-cc4398eae089 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.405678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a16a56-8a60-4983-87cf-cc4398eae089 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.405954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a16a56-8a60-4983-87cf-cc4398eae089 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.406254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a16a56-8a60-4983-87cf-cc4398eae089 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.406878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a16a56-8a60-4983-87cf-cc4398eae089 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:46.407177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a16a56-8a60-4983-87cf-cc4398eae089 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:46.408852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a16a56-8a60-4983-87cf-cc4398eae089 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:46.409745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a449036-f907-475d-975a-86fcf41b00f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.410382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a449036-f907-475d-975a-86fcf41b00f9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.410772 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a449036-f907-475d-975a-86fcf41b00f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.411050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a449036-f907-475d-975a-86fcf41b00f9 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.411743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a449036-f907-475d-975a-86fcf41b00f9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:46.412047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a449036-f907-475d-975a-86fcf41b00f9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:46.413463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a449036-f907-475d-975a-86fcf41b00f9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:46.419390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 436/855] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:46.419390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a16a56-8a60-4983-87cf-cc4398eae089 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:46.434103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 420/856] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:46.435097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a449036-f907-475d-975a-86fcf41b00f9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:46.521341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a16a56-8a60-4983-87cf-cc4398eae089 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:46.539352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a449036-f907-475d-975a-86fcf41b00f9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:46.557749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686415e-f2fe-43e3-b8ff-5b816868b2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.558878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686415e-f2fe-43e3-b8ff-5b816868b2a4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.559237 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686415e-f2fe-43e3-b8ff-5b816868b2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.559433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c686415e-f2fe-43e3-b8ff-5b816868b2a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.571768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c686415e-f2fe-43e3-b8ff-5b816868b2a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c686415e-f2fe-43e3-b8ff-5b816868b2a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:46.577288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:46.578251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:46.579104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 437/857] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:46.580457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686415e-f2fe-43e3-b8ff-5b816868b2a4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:46.580805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bff48-95e8-4a0a-9701-7807d6961f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.581423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bff48-95e8-4a0a-9701-7807d6961f91 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.581695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bff48-95e8-4a0a-9701-7807d6961f91 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.581960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68bff48-95e8-4a0a-9701-7807d6961f91 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.582634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bff48-95e8-4a0a-9701-7807d6961f91 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:46.582931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bff48-95e8-4a0a-9701-7807d6961f91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:46.584567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68bff48-95e8-4a0a-9701-7807d6961f91 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:46.589371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 421/858] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:46.590193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bff48-95e8-4a0a-9701-7807d6961f91 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:46.621843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:46.644304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:46.727823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68bff48-95e8-4a0a-9701-7807d6961f91 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:46.732777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c686415e-f2fe-43e3-b8ff-5b816868b2a4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:46.766437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d0287-294c-4de3-a3d5-70eb0204526c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.767118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d0287-294c-4de3-a3d5-70eb0204526c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.767402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d0287-294c-4de3-a3d5-70eb0204526c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.767673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2d0287-294c-4de3-a3d5-70eb0204526c None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.768339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d0287-294c-4de3-a3d5-70eb0204526c None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:46.768640 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d0287-294c-4de3-a3d5-70eb0204526c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:46.770032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2d0287-294c-4de3-a3d5-70eb0204526c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:46.779758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 422/859] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:46.780572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae73ee1-7b21-4ade-8208-9273c7294edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.781264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae73ee1-7b21-4ade-8208-9273c7294edd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.781553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae73ee1-7b21-4ade-8208-9273c7294edd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.781827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae73ee1-7b21-4ade-8208-9273c7294edd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.782593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae73ee1-7b21-4ade-8208-9273c7294edd None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=4815ef133a3a4604bc467c222d450fbf) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:46.782899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae73ee1-7b21-4ade-8208-9273c7294edd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:46.784345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae73ee1-7b21-4ade-8208-9273c7294edd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:46.786823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d0287-294c-4de3-a3d5-70eb0204526c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:46.797780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 438/860] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/role_assignments?user.id=4815ef133a3a4604bc467c222d450fbf&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 608 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:46.897036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:46.911229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2d0287-294c-4de3-a3d5-70eb0204526c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:46.950518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63a999-d079-4082-8daa-fafece9c5ea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:46.951186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63a999-d079-4082-8daa-fafece9c5ea7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:46.951467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63a999-d079-4082-8daa-fafece9c5ea7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:46.951740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a63a999-d079-4082-8daa-fafece9c5ea7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:46.965077 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63a999-d079-4082-8daa-fafece9c5ea7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:46.966604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a63a999-d079-4082-8daa-fafece9c5ea7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9a63a999-d079-4082-8daa-fafece9c5ea7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:46.982506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:46.983516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:46.984408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:46.984408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:46.984408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:46.984408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 423/861] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:46.991196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae73ee1-7b21-4ade-8208-9273c7294edd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:47.085262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:47.098130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae73ee1-7b21-4ade-8208-9273c7294edd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:47.134510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adb415-a5d4-4577-ba58-9d1f6ae12a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:47.135132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adb415-a5d4-4577-ba58-9d1f6ae12a37 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:47.135413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adb415-a5d4-4577-ba58-9d1f6ae12a37 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:47.135697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9adb415-a5d4-4577-ba58-9d1f6ae12a37 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:47.136403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adb415-a5d4-4577-ba58-9d1f6ae12a37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:47.136715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adb415-a5d4-4577-ba58-9d1f6ae12a37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:47.138260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9adb415-a5d4-4577-ba58-9d1f6ae12a37 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:47.156798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 439/862] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:47.158134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63a999-d079-4082-8daa-fafece9c5ea7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:47.265065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a63a999-d079-4082-8daa-fafece9c5ea7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:47.301720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c66480-81a5-494d-ba92-a5ecf89c06a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:47.302413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c66480-81a5-494d-ba92-a5ecf89c06a7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:47.302688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c66480-81a5-494d-ba92-a5ecf89c06a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:47.302958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c66480-81a5-494d-ba92-a5ecf89c06a7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:47.311859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c66480-81a5-494d-ba92-a5ecf89c06a7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:47.313316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c66480-81a5-494d-ba92-a5ecf89c06a7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:47.316000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-81c66480-81a5-494d-ba92-a5ecf89c06a7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:47.326673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:47.328918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:47.331152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:47.332461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 424/863] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:48:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:47.335430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adb415-a5d4-4577-ba58-9d1f6ae12a37 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:47.444789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9adb415-a5d4-4577-ba58-9d1f6ae12a37 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:47.482888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e2f418-39f0-4c17-b4fb-79af38c86c46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:47.483854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2f418-39f0-4c17-b4fb-79af38c86c46 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:47.484346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2f418-39f0-4c17-b4fb-79af38c86c46 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:47.484840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e2f418-39f0-4c17-b4fb-79af38c86c46 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:47.485794 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2f418-39f0-4c17-b4fb-79af38c86c46 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:47.486331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2f418-39f0-4c17-b4fb-79af38c86c46 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:47.487967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e2f418-39f0-4c17-b4fb-79af38c86c46 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:47.507073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 440/864] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Sep 13 06:48:47 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:47.518166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c66480-81a5-494d-ba92-a5ecf89c06a7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:47.628036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c66480-81a5-494d-ba92-a5ecf89c06a7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:47.651536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa9007-24ac-4ea8-9fef-6c439bf1ce4d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:47.652564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa9007-24ac-4ea8-9fef-6c439bf1ce4d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:47.653136 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa9007-24ac-4ea8-9fef-6c439bf1ce4d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:47.654996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 441/865] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:47.660257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe25c00-7a02-404b-8b67-6431238f635f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:47.660777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe25c00-7a02-404b-8b67-6431238f635f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:47.661251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe25c00-7a02-404b-8b67-6431238f635f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:47.665594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffa688-f194-48c0-ba69-d00f9f48359e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:47.666248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffa688-f194-48c0-ba69-d00f9f48359e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:47.666528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffa688-f194-48c0-ba69-d00f9f48359e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:47.666801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffa688-f194-48c0-ba69-d00f9f48359e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:47.675427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:47.683128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5fe25c00-7a02-404b-8b67-6431238f635f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:47.685710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5fe25c00-7a02-404b-8b67-6431238f635f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:47.737678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:47.738745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffa688-f194-48c0-ba69-d00f9f48359e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:47.740447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffa688-f194-48c0-ba69-d00f9f48359e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:47.761080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 442/866] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:47.857662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 425/867] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:47 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:48:47.859142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe25c00-7a02-404b-8b67-6431238f635f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:47.968345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe25c00-7a02-404b-8b67-6431238f635f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:48.006296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6a388-bdc8-408f-94aa-5670ab3cecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:48.006861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6a388-bdc8-408f-94aa-5670ab3cecd3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:48.007171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6a388-bdc8-408f-94aa-5670ab3cecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:48.007441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa6a388-bdc8-408f-94aa-5670ab3cecd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:48.010197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:48.018695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6a388-bdc8-408f-94aa-5670ab3cecd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:48.020756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa6a388-bdc8-408f-94aa-5670ab3cecd3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ffa6a388-bdc8-408f-94aa-5670ab3cecd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:48.035985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:48.036954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:48.037989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:48.037989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:48.037989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:48.037989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 443/868] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:48.044488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffa688-f194-48c0-ba69-d00f9f48359e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:48.045926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6a388-bdc8-408f-94aa-5670ab3cecd3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:48.146442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:48.149792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffa688-f194-48c0-ba69-d00f9f48359e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:48.154213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa6a388-bdc8-408f-94aa-5670ab3cecd3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:48.187872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d327f2-a47f-4022-b384-ac5baa78a52c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:48.189041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d327f2-a47f-4022-b384-ac5baa78a52c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:48.189518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d327f2-a47f-4022-b384-ac5baa78a52c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:48.189804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695044d-ff4c-44d2-9310-e07fbeb5cac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:48.190276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d327f2-a47f-4022-b384-ac5baa78a52c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:48.190448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695044d-ff4c-44d2-9310-e07fbeb5cac4 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:48.190732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695044d-ff4c-44d2-9310-e07fbeb5cac4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:48.191007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695044d-ff4c-44d2-9310-e07fbeb5cac4 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:48.191682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695044d-ff4c-44d2-9310-e07fbeb5cac4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:48.191903 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d327f2-a47f-4022-b384-ac5baa78a52c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:48.192397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d327f2-a47f-4022-b384-ac5baa78a52c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:48.193072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f695044d-ff4c-44d2-9310-e07fbeb5cac4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:48.194172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d327f2-a47f-4022-b384-ac5baa78a52c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:48.212547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 426/869] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:48.214157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d327f2-a47f-4022-b384-ac5baa78a52c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:48.221022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 444/870] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:48:48 2022] POST /identity/v3/services => generated 251 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:48:48.328479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d327f2-a47f-4022-b384-ac5baa78a52c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:48.373132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c7ad-36c9-4cd0-a105-4805dd2d1b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:48.374216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c7ad-36c9-4cd0-a105-4805dd2d1b8c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:48.374694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c7ad-36c9-4cd0-a105-4805dd2d1b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:48.375159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee8c7ad-36c9-4cd0-a105-4805dd2d1b8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:48.383292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8c7ad-36c9-4cd0-a105-4805dd2d1b8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:48.385099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee8c7ad-36c9-4cd0-a105-4805dd2d1b8c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4ee8c7ad-36c9-4cd0-a105-4805dd2d1b8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:48.397656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:48.398633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:48.399555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:48.399555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:48.399555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:48.399555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:48.399555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:48.399555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:48.399555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:48.399555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:48.406198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 427/871] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:48.408188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695044d-ff4c-44d2-9310-e07fbeb5cac4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:48.460025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:48.515952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f695044d-ff4c-44d2-9310-e07fbeb5cac4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:48.551017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1175dc08-1568-4c19-ba62-0121030e6d4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:48.552030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175dc08-1568-4c19-ba62-0121030e6d4a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:48.552525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175dc08-1568-4c19-ba62-0121030e6d4a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:48.552997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1175dc08-1568-4c19-ba62-0121030e6d4a None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:48.553895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175dc08-1568-4c19-ba62-0121030e6d4a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:48.554426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175dc08-1568-4c19-ba62-0121030e6d4a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:48.556136 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1175dc08-1568-4c19-ba62-0121030e6d4a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:48.573793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 445/872] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:48.576292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c7ad-36c9-4cd0-a105-4805dd2d1b8c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:48.614498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:48.681521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee8c7ad-36c9-4cd0-a105-4805dd2d1b8c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:48.721026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a82a1c0-5128-445a-971f-10a2442e5681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:48.721664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82a1c0-5128-445a-971f-10a2442e5681 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:48.721944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82a1c0-5128-445a-971f-10a2442e5681 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:48.722252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a82a1c0-5128-445a-971f-10a2442e5681 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:48.723000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82a1c0-5128-445a-971f-10a2442e5681 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=7363637f44264a65b66ba53da825eaff) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:48.723308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82a1c0-5128-445a-971f-10a2442e5681 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:48.724803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a82a1c0-5128-445a-971f-10a2442e5681 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:48.738247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 428/873] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:48 2022] GET /identity/v3/role_assignments?user.id=7363637f44264a65b66ba53da825eaff&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:48.821886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b264d-b5cd-4f8e-883f-5e38b1c1857b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:48.822481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b264d-b5cd-4f8e-883f-5e38b1c1857b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:48.822573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59b264d-b5cd-4f8e-883f-5e38b1c1857b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:48.824273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 446/874] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:48.832100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e0a90-4a51-4f3c-998b-fbf9337fcadb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:48.832505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e0a90-4a51-4f3c-998b-fbf9337fcadb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:48.832670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288e0a90-4a51-4f3c-998b-fbf9337fcadb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:48.852192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-288e0a90-4a51-4f3c-998b-fbf9337fcadb None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:48.853415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-288e0a90-4a51-4f3c-998b-fbf9337fcadb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:48.927066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 429/875] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:48 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:49.157249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7277e-30ed-4c4f-8451-700bbbe0b556 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:49.158135 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7277e-30ed-4c4f-8451-700bbbe0b556 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:49.158652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e7277e-30ed-4c4f-8451-700bbbe0b556 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:49.160617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 447/876] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:49.166830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe5ff1-705a-46e7-945c-064ef051df89 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:49.168025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe5ff1-705a-46e7-945c-064ef051df89 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:49.168663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bbe5ff1-705a-46e7-945c-064ef051df89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:49.188238 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8bbe5ff1-705a-46e7-945c-064ef051df89 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:49.189761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8bbe5ff1-705a-46e7-945c-064ef051df89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:49.192963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:49.200254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7277e-30ed-4c4f-8451-700bbbe0b556 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:49.270341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 430/877] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:49 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:49.319835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e7277e-30ed-4c4f-8451-700bbbe0b556 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:49.329448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:49.355447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b481d8-3c86-4b87-af5d-11eb92e784b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:49.356099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b481d8-3c86-4b87-af5d-11eb92e784b0 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:49.356374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b481d8-3c86-4b87-af5d-11eb92e784b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:49.356644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b481d8-3c86-4b87-af5d-11eb92e784b0 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:49.357316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b481d8-3c86-4b87-af5d-11eb92e784b0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:49.358812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b481d8-3c86-4b87-af5d-11eb92e784b0 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:49.382485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 448/878] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:48:49 2022] POST /identity/v3/services => generated 269 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:49.544623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe5ff1-705a-46e7-945c-064ef051df89 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:49.652814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bbe5ff1-705a-46e7-945c-064ef051df89 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:49.693168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e0c47-c67d-471e-b512-314595b87eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:49.694282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e0c47-c67d-471e-b512-314595b87eef None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:49.694759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e0c47-c67d-471e-b512-314595b87eef None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:49.695273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e0c47-c67d-471e-b512-314595b87eef None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:49.696099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e0c47-c67d-471e-b512-314595b87eef None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:49.697753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e0c47-c67d-471e-b512-314595b87eef None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:49.701771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-734e0c47-c67d-471e-b512-314595b87eef None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:49.708982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:49.710094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:49.710968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 431/879] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:48:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:49.719016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b481d8-3c86-4b87-af5d-11eb92e784b0 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:49.823497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b481d8-3c86-4b87-af5d-11eb92e784b0 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:49.858577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7653b8e-8234-4671-8285-569aa6681e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:49.859195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7653b8e-8234-4671-8285-569aa6681e80 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:49.859470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7653b8e-8234-4671-8285-569aa6681e80 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:49.859738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7653b8e-8234-4671-8285-569aa6681e80 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:49.860366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7653b8e-8234-4671-8285-569aa6681e80 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:49.860667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7653b8e-8234-4671-8285-569aa6681e80 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:49.862039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7653b8e-8234-4671-8285-569aa6681e80 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:49.867143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 449/880] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:48:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:49.872829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e0c47-c67d-471e-b512-314595b87eef None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:49.984847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e0c47-c67d-471e-b512-314595b87eef None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:50.020294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78812a3e-25eb-47bb-825d-ad4c0e672d84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:50.020908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78812a3e-25eb-47bb-825d-ad4c0e672d84 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:50.021181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78812a3e-25eb-47bb-825d-ad4c0e672d84 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:50.021453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78812a3e-25eb-47bb-825d-ad4c0e672d84 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:50.022102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78812a3e-25eb-47bb-825d-ad4c0e672d84 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:50.022414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78812a3e-25eb-47bb-825d-ad4c0e672d84 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:50.023783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78812a3e-25eb-47bb-825d-ad4c0e672d84 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:50.028733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 432/881] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:48:49 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:50.549178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcb2d9-0c05-4418-a5f1-9827dccff627 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:50.549751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcb2d9-0c05-4418-a5f1-9827dccff627 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:50.550028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fcb2d9-0c05-4418-a5f1-9827dccff627 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:50.551691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 450/882] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:50.557127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9651232-2520-46d9-9db0-35d478288c68 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:50.557589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9651232-2520-46d9-9db0-35d478288c68 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:50.557683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9651232-2520-46d9-9db0-35d478288c68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:50.577118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b9651232-2520-46d9-9db0-35d478288c68 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:50.578337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b9651232-2520-46d9-9db0-35d478288c68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:50.651611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:50.660995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 433/883] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:50.927296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcb2d9-0c05-4418-a5f1-9827dccff627 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:50.962325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:50.971549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e6a3e-7d5e-40a0-91f9-58b34dd253f7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:50.971848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e6a3e-7d5e-40a0-91f9-58b34dd253f7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:50.972098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11e6a3e-7d5e-40a0-91f9-58b34dd253f7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:50.973611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 434/884] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:50.978678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ac165-e4d6-4d10-8df5-10759a82e83b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:50.978961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ac165-e4d6-4d10-8df5-10759a82e83b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:50.979231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8ac165-e4d6-4d10-8df5-10759a82e83b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:50.997566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3b8ac165-e4d6-4d10-8df5-10759a82e83b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:50.998773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3b8ac165-e4d6-4d10-8df5-10759a82e83b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:51.045829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fcb2d9-0c05-4418-a5f1-9827dccff627 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:51.082183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b02bb9-b22a-4591-97a9-a257718446d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:51.082819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b02bb9-b22a-4591-97a9-a257718446d4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.083092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b02bb9-b22a-4591-97a9-a257718446d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.083360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b02bb9-b22a-4591-97a9-a257718446d4 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:51.083995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b02bb9-b22a-4591-97a9-a257718446d4 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:51.085372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b02bb9-b22a-4591-97a9-a257718446d4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:51.086498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 435/885] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:50 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-33b02bb9-b22a-4591-97a9-a257718446d4 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:51.098545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:51.099557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 06:48:51.100400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:51.103092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 451/886] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:51.110101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ac165-e4d6-4d10-8df5-10759a82e83b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:51.210282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8ac165-e4d6-4d10-8df5-10759a82e83b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:51.234370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:51.247203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dd54b-1f13-4178-8c82-577ab67d6687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:51.247825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dd54b-1f13-4178-8c82-577ab67d6687 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.248101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dd54b-1f13-4178-8c82-577ab67d6687 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.248372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9dd54b-1f13-4178-8c82-577ab67d6687 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:51.249002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dd54b-1f13-4178-8c82-577ab67d6687 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:51.249305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dd54b-1f13-4178-8c82-577ab67d6687 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:51.250722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9dd54b-1f13-4178-8c82-577ab67d6687 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:51.255876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 436/887] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:51.261638 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b02bb9-b22a-4591-97a9-a257718446d4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:51.359925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:51.365927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b02bb9-b22a-4591-97a9-a257718446d4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:51.395143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dd54b-1f13-4178-8c82-577ab67d6687 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:51.406731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1155df8-f1e0-4a8e-9cfe-7657136b87b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:51.407362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1155df8-f1e0-4a8e-9cfe-7657136b87b7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.407638 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1155df8-f1e0-4a8e-9cfe-7657136b87b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.408346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1155df8-f1e0-4a8e-9cfe-7657136b87b7 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:51.408987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1155df8-f1e0-4a8e-9cfe-7657136b87b7 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:51.409295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1155df8-f1e0-4a8e-9cfe-7657136b87b7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:51.411149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1155df8-f1e0-4a8e-9cfe-7657136b87b7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:51.418556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 452/888] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:51.424180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1155df8-f1e0-4a8e-9cfe-7657136b87b7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:51.535676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1155df8-f1e0-4a8e-9cfe-7657136b87b7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:51.541961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9dd54b-1f13-4178-8c82-577ab67d6687 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:51.576461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb8798-5581-4cf1-b992-f96ae883c18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:51.577093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb8798-5581-4cf1-b992-f96ae883c18e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.577367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb8798-5581-4cf1-b992-f96ae883c18e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.577636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb8798-5581-4cf1-b992-f96ae883c18e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:51.578381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb8798-5581-4cf1-b992-f96ae883c18e None admin] RBAC: Adding query filter params (interface=public, service_id=62f744bd60c14338a714ac8d6c970de7, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:51.578753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb8798-5581-4cf1-b992-f96ae883c18e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:51.580163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb8798-5581-4cf1-b992-f96ae883c18e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:51.591473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 453/889] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Sep 13 06:48:51 2022] GET /identity/v3/endpoints?service_id=62f744bd60c14338a714ac8d6c970de7&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:51.594037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb8798-5581-4cf1-b992-f96ae883c18e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:51.599175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ab322-d0fd-4c8d-aee1-24ac3935815f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:51.599831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ab322-d0fd-4c8d-aee1-24ac3935815f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.600110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ab322-d0fd-4c8d-aee1-24ac3935815f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.600379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0ab322-d0fd-4c8d-aee1-24ac3935815f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:51.619329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b0ab322-d0fd-4c8d-aee1-24ac3935815f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-7b0ab322-d0fd-4c8d-aee1-24ac3935815f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:51.625643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:51.626608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:48:51.627460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:51.628872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:51.629931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 437/890] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Sep 13 06:48:51 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:51.636048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ab322-d0fd-4c8d-aee1-24ac3935815f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:51.694871 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:51.715882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb8798-5581-4cf1-b992-f96ae883c18e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:51.749354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0ab322-d0fd-4c8d-aee1-24ac3935815f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:51.755339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470b72e3-1df0-4eba-8356-3f37fb374337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:51.755953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b72e3-1df0-4eba-8356-3f37fb374337 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.756228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b72e3-1df0-4eba-8356-3f37fb374337 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.756496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470b72e3-1df0-4eba-8356-3f37fb374337 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:51.757103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b72e3-1df0-4eba-8356-3f37fb374337 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:51.758509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470b72e3-1df0-4eba-8356-3f37fb374337 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:51.773232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 454/891] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Sep 13 06:48:51 2022] GET /identity/v3/services => generated 1268 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 06:48:51.774597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8d528-b2df-472f-9244-2183951f9845 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.774879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8d528-b2df-472f-9244-2183951f9845 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.775153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e8d528-b2df-472f-9244-2183951f9845 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:51.776709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 455/892] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:51.782431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1881c-f355-4e4e-b2d4-cac039a29407 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.782848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1881c-f355-4e4e-b2d4-cac039a29407 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.783120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a1881c-f355-4e4e-b2d4-cac039a29407 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:51.794971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde32994-3995-489c-8641-64574d0481d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:51.795951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde32994-3995-489c-8641-64574d0481d6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.796406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde32994-3995-489c-8641-64574d0481d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.796879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde32994-3995-489c-8641-64574d0481d6 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:51.797575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde32994-3995-489c-8641-64574d0481d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:51.797879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde32994-3995-489c-8641-64574d0481d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:51.799354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde32994-3995-489c-8641-64574d0481d6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:51.803603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-74a1881c-f355-4e4e-b2d4-cac039a29407 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:51.805171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-74a1881c-f355-4e4e-b2d4-cac039a29407 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:51.809403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 438/893] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Sep 13 06:48:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:51.819662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde32994-3995-489c-8641-64574d0481d6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:51.890319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 456/894] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:51 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:48:51.950995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde32994-3995-489c-8641-64574d0481d6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:51.961735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:51.987279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c6702-042a-45d2-8106-576fe6fc0691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:51.988260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c6702-042a-45d2-8106-576fe6fc0691 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:51.988538 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c6702-042a-45d2-8106-576fe6fc0691 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:51.988815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503c6702-042a-45d2-8106-576fe6fc0691 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.004502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c6702-042a-45d2-8106-576fe6fc0691 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:52.005955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503c6702-042a-45d2-8106-576fe6fc0691 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-503c6702-042a-45d2-8106-576fe6fc0691 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:52.022776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:52.023822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:52.024740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:52.024740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:52.024740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:52.024740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 439/895] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:52.031470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1881c-f355-4e4e-b2d4-cac039a29407 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:52.070635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:52.146757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a1881c-f355-4e4e-b2d4-cac039a29407 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:52.184251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49605b1-ea9c-40c7-bc4d-007c77d0fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:52.185445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49605b1-ea9c-40c7-bc4d-007c77d0fc44 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.185942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49605b1-ea9c-40c7-bc4d-007c77d0fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.186046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c6702-042a-45d2-8106-576fe6fc0691 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:52.186537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49605b1-ea9c-40c7-bc4d-007c77d0fc44 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.187420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49605b1-ea9c-40c7-bc4d-007c77d0fc44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:52.187919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49605b1-ea9c-40c7-bc4d-007c77d0fc44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:52.190919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49605b1-ea9c-40c7-bc4d-007c77d0fc44 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:52.207541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 457/896] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:52.212131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3581697-97e2-48a4-b509-e88575b20b0c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.212131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3581697-97e2-48a4-b509-e88575b20b0c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.212131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3581697-97e2-48a4-b509-e88575b20b0c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.212393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 458/897] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:52.213026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3581697-97e2-48a4-b509-e88575b20b0c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:52.295081 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503c6702-042a-45d2-8106-576fe6fc0691 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:52.302333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:52.326528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3581697-97e2-48a4-b509-e88575b20b0c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:52.337282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5481d027-17e3-481b-915b-d05f5b7977c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:52.337914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481d027-17e3-481b-915b-d05f5b7977c4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.338534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481d027-17e3-481b-915b-d05f5b7977c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.338815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5481d027-17e3-481b-915b-d05f5b7977c4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.339794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481d027-17e3-481b-915b-d05f5b7977c4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:52.341184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5481d027-17e3-481b-915b-d05f5b7977c4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-5481d027-17e3-481b-915b-d05f5b7977c4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:52.349944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:52.350921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:52.352496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 440/898] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Sep 13 06:48:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:52.353284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50b657-fe29-4155-9244-a6be3cb2f746 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.353284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50b657-fe29-4155-9244-a6be3cb2f746 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.353284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d50b657-fe29-4155-9244-a6be3cb2f746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.368365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e36a4-aa99-4dae-9442-d66c82977f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:52.369000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e36a4-aa99-4dae-9442-d66c82977f10 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.369275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e36a4-aa99-4dae-9442-d66c82977f10 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.369542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2e36a4-aa99-4dae-9442-d66c82977f10 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.372916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5d50b657-fe29-4155-9244-a6be3cb2f746 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:52.374347 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5d50b657-fe29-4155-9244-a6be3cb2f746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:52.389918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e36a4-aa99-4dae-9442-d66c82977f10 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:52.391727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2e36a4-aa99-4dae-9442-d66c82977f10 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:52.397615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ea2e36a4-aa99-4dae-9442-d66c82977f10 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:52.414263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:52.415450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:52.416542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:52.421010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 459/899] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:48:52 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:52.421772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e36a4-aa99-4dae-9442-d66c82977f10 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:52.456977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 441/900] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:52 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:52.457744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50b657-fe29-4155-9244-a6be3cb2f746 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:52.565535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2e36a4-aa99-4dae-9442-d66c82977f10 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:52.571846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d50b657-fe29-4155-9244-a6be3cb2f746 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:52.614116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696a3ea-d26f-4b8f-8231-02d5ae68c55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:52.614116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696a3ea-d26f-4b8f-8231-02d5ae68c55f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.614116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696a3ea-d26f-4b8f-8231-02d5ae68c55f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.614116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3696a3ea-d26f-4b8f-8231-02d5ae68c55f None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.615047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696a3ea-d26f-4b8f-8231-02d5ae68c55f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:52.615047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696a3ea-d26f-4b8f-8231-02d5ae68c55f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:52.615207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aabd9-7e4e-4356-8daa-2d4eaebe0949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:52.615317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3696a3ea-d26f-4b8f-8231-02d5ae68c55f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:52.615573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aabd9-7e4e-4356-8daa-2d4eaebe0949 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.615952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aabd9-7e4e-4356-8daa-2d4eaebe0949 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.616571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6aabd9-7e4e-4356-8daa-2d4eaebe0949 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.626114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aabd9-7e4e-4356-8daa-2d4eaebe0949 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:52.626114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aabd9-7e4e-4356-8daa-2d4eaebe0949 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:52.626114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6aabd9-7e4e-4356-8daa-2d4eaebe0949 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:52.627230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 460/901] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:48:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:52.632590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aabd9-7e4e-4356-8daa-2d4eaebe0949 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:52.635579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:52.645484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 442/902] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Sep 13 06:48:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:52.651072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696a3ea-d26f-4b8f-8231-02d5ae68c55f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:52.669610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:52.750230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6aabd9-7e4e-4356-8daa-2d4eaebe0949 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:52.772844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3696a3ea-d26f-4b8f-8231-02d5ae68c55f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:52.791331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceefa73-a5d4-4486-b32a-f865566cbed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:52.791948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceefa73-a5d4-4486-b32a-f865566cbed5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.792221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceefa73-a5d4-4486-b32a-f865566cbed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.792605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceefa73-a5d4-4486-b32a-f865566cbed5 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.793339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceefa73-a5d4-4486-b32a-f865566cbed5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:52.793640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceefa73-a5d4-4486-b32a-f865566cbed5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:52.795525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ceefa73-a5d4-4486-b32a-f865566cbed5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:52.805891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 461/903] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:48:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:52.805891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceefa73-a5d4-4486-b32a-f865566cbed5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:52.810978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a4a3e-bb6a-4d66-b404-d2f91fa204a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:52.811599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4a3e-bb6a-4d66-b404-d2f91fa204a2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.811858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4a3e-bb6a-4d66-b404-d2f91fa204a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.812130 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211a4a3e-bb6a-4d66-b404-d2f91fa204a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.875618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:52.882321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a4a3e-bb6a-4d66-b404-d2f91fa204a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:52.883775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-211a4a3e-bb6a-4d66-b404-d2f91fa204a2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:52.897899 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:52.917023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceefa73-a5d4-4486-b32a-f865566cbed5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:52.953858 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a6a53-1663-4a29-abb9-905e81e74472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:52.954513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a6a53-1663-4a29-abb9-905e81e74472 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:52.954820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a6a53-1663-4a29-abb9-905e81e74472 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:52.955072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a6a53-1663-4a29-abb9-905e81e74472 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:52.955728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a6a53-1663-4a29-abb9-905e81e74472 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:52.957113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a6a53-1663-4a29-abb9-905e81e74472 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:52.982370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 462/904] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:48:52 2022] POST /identity/v3/services => generated 241 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:52.983453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a6a53-1663-4a29-abb9-905e81e74472 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:52.995723 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 443/905] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:48:52 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:48:52.999998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a4a3e-bb6a-4d66-b404-d2f91fa204a2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:53.127632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211a4a3e-bb6a-4d66-b404-d2f91fa204a2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:53.148398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a6a53-1663-4a29-abb9-905e81e74472 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:53.150620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:53.174617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:53.176605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d7f4e-9892-4f73-978a-c03afe338ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:53.176963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d7f4e-9892-4f73-978a-c03afe338ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:53.177232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d7f4e-9892-4f73-978a-c03afe338ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:53.177503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d7f4e-9892-4f73-978a-c03afe338ff2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:53.191389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d7f4e-9892-4f73-978a-c03afe338ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:53.193580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68d7f4e-9892-4f73-978a-c03afe338ff2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:53.193783 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968cebb-25a6-4bf0-9da6-6175e7dd823f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:53.194418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968cebb-25a6-4bf0-9da6-6175e7dd823f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:53.194682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968cebb-25a6-4bf0-9da6-6175e7dd823f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:53.195044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7968cebb-25a6-4bf0-9da6-6175e7dd823f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:53.195806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968cebb-25a6-4bf0-9da6-6175e7dd823f None admin] RBAC: Adding query filter params (interface=public, service_id=117fb14f4a894d1e98a14bd2f455e173, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:53.196110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968cebb-25a6-4bf0-9da6-6175e7dd823f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:53.197736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7968cebb-25a6-4bf0-9da6-6175e7dd823f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:53.206131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 463/906] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Sep 13 06:48:52 2022] GET /identity/v3/endpoints?service_id=117fb14f4a894d1e98a14bd2f455e173&interface=public®ion_id=RegionOne => generated 194 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:53.212602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968cebb-25a6-4bf0-9da6-6175e7dd823f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b68d7f4e-9892-4f73-978a-c03afe338ff2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:53.212734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:53.213615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:53.214688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:53.214688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:53.214688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:53.214688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 444/907] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:53.220920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d7f4e-9892-4f73-978a-c03afe338ff2 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:53.333160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7968cebb-25a6-4bf0-9da6-6175e7dd823f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:53.338367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68d7f4e-9892-4f73-978a-c03afe338ff2 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:53.368747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285ac44-f641-4254-b111-546349cfca19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:53.369366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ac44-f641-4254-b111-546349cfca19 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:53.369641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ac44-f641-4254-b111-546349cfca19 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:53.369931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0285ac44-f641-4254-b111-546349cfca19 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:53.370589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285ac44-f641-4254-b111-546349cfca19 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:53.372016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0285ac44-f641-4254-b111-546349cfca19 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:53.386297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 464/908] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Sep 13 06:48:53 2022] GET /identity/v3/services => generated 1497 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 06:48:53.386580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2d7b2-b78e-405e-998f-48f6a28b38de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:53.386580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2d7b2-b78e-405e-998f-48f6a28b38de None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:53.386580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2d7b2-b78e-405e-998f-48f6a28b38de None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:53.386580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2d7b2-b78e-405e-998f-48f6a28b38de None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:53.386897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2d7b2-b78e-405e-998f-48f6a28b38de None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:53.386897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2d7b2-b78e-405e-998f-48f6a28b38de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:53.386897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2d7b2-b78e-405e-998f-48f6a28b38de None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:53.400849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 445/909] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:53.406430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285ac44-f641-4254-b111-546349cfca19 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:53.462648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:53.524089 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0285ac44-f641-4254-b111-546349cfca19 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:53.569391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d168c-68b6-4635-a9e2-7835766b572c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:53.570494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d168c-68b6-4635-a9e2-7835766b572c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:53.571059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d168c-68b6-4635-a9e2-7835766b572c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:53.571537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d168c-68b6-4635-a9e2-7835766b572c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:53.579892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d168c-68b6-4635-a9e2-7835766b572c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:53.581372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d168c-68b6-4635-a9e2-7835766b572c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6c2d168c-68b6-4635-a9e2-7835766b572c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:53.595551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:53.596478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:53.597381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:53.597381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:53.597381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:53.597381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:53.597381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:53.597381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:53.597381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:53.597381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:53.597381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 465/910] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:53.601476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2d7b2-b78e-405e-998f-48f6a28b38de None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:53.637041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:53.714712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2d7b2-b78e-405e-998f-48f6a28b38de None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:53.752231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d55b44-7630-416b-9a63-7918c15e356d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:53.752849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d55b44-7630-416b-9a63-7918c15e356d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:53.753124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d55b44-7630-416b-9a63-7918c15e356d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:53.753388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d55b44-7630-416b-9a63-7918c15e356d None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:53.754074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d55b44-7630-416b-9a63-7918c15e356d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:53.754407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d55b44-7630-416b-9a63-7918c15e356d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:53.755846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d55b44-7630-416b-9a63-7918c15e356d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:53.768490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 446/911] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:53.774152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d168c-68b6-4635-a9e2-7835766b572c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:53.878998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d168c-68b6-4635-a9e2-7835766b572c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:53.894593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:53.922636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d058246-57f1-40b7-9335-09ad0b1bc0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:53.923267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058246-57f1-40b7-9335-09ad0b1bc0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:53.923544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058246-57f1-40b7-9335-09ad0b1bc0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:53.923817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d058246-57f1-40b7-9335-09ad0b1bc0d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:53.924560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d058246-57f1-40b7-9335-09ad0b1bc0d3 None admin] RBAC: Adding query filter params (role.id=d0cb30355e064a3d8562073f93479915, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=7363637f44264a65b66ba53da825eaff) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:53.924869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d058246-57f1-40b7-9335-09ad0b1bc0d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:53.926374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d058246-57f1-40b7-9335-09ad0b1bc0d3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:53.933540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 466/912] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:48:53 2022] GET /identity/v3/role_assignments?user.id=7363637f44264a65b66ba53da825eaff&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=d0cb30355e064a3d8562073f93479915 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:54.031220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec9be9-8747-4d1b-8f0c-4b3b93750c06 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:54.031891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec9be9-8747-4d1b-8f0c-4b3b93750c06 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:54.032365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cec9be9-8747-4d1b-8f0c-4b3b93750c06 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:54.034402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 447/913] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:54.040418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3a194-7224-49a9-9879-3d42f082c15f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:54.041096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3a194-7224-49a9-9879-3d42f082c15f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:54.041564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff3a194-7224-49a9-9879-3d42f082c15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:54.062152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6ff3a194-7224-49a9-9879-3d42f082c15f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:54.063389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6ff3a194-7224-49a9-9879-3d42f082c15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:54.138810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 467/914] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:54 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:54.410159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec9be9-8747-4d1b-8f0c-4b3b93750c06 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:54.500266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:54.517596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cec9be9-8747-4d1b-8f0c-4b3b93750c06 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:54.551684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cf1e3-fb44-4f48-9e9c-ed9fa00e1c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:54.552610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cf1e3-fb44-4f48-9e9c-ed9fa00e1c4b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:54.553112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cf1e3-fb44-4f48-9e9c-ed9fa00e1c4b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:54.553608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6cf1e3-fb44-4f48-9e9c-ed9fa00e1c4b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:54.554406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cf1e3-fb44-4f48-9e9c-ed9fa00e1c4b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:54.555962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6cf1e3-fb44-4f48-9e9c-ed9fa00e1c4b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bf6cf1e3-fb44-4f48-9e9c-ed9fa00e1c4b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:54.565602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:54.566617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 13 06:48:54.567474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:54.570238 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 448/915] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:54.576565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3a194-7224-49a9-9879-3d42f082c15f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:54.689702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff3a194-7224-49a9-9879-3d42f082c15f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:54.691999 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:54.724704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be78997-6fea-4f7b-ad6c-0534f33e22a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:54.725373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be78997-6fea-4f7b-ad6c-0534f33e22a1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:54.725654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be78997-6fea-4f7b-ad6c-0534f33e22a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:54.725923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be78997-6fea-4f7b-ad6c-0534f33e22a1 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:54.726578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be78997-6fea-4f7b-ad6c-0534f33e22a1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:54.726881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be78997-6fea-4f7b-ad6c-0534f33e22a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:54.728258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be78997-6fea-4f7b-ad6c-0534f33e22a1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:54.737657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 468/916] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:54.743197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cf1e3-fb44-4f48-9e9c-ed9fa00e1c4b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:54.851005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6cf1e3-fb44-4f48-9e9c-ed9fa00e1c4b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:54.889416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af649430-8909-4e18-8d0e-3758a01c0921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:54.890106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af649430-8909-4e18-8d0e-3758a01c0921 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:54.890398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af649430-8909-4e18-8d0e-3758a01c0921 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:54.890676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af649430-8909-4e18-8d0e-3758a01c0921 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:54.891309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af649430-8909-4e18-8d0e-3758a01c0921 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:54.891620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af649430-8909-4e18-8d0e-3758a01c0921 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:54.893009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af649430-8909-4e18-8d0e-3758a01c0921 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:54.898363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 449/917] 10.222.0.25 () {56 vars in 1114 bytes} [Tue Sep 13 06:48:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:54.904246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be78997-6fea-4f7b-ad6c-0534f33e22a1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:55.003150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be78997-6fea-4f7b-ad6c-0534f33e22a1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:55.037273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dbc6af4-313f-4a68-845d-cd8d2c71e002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:55.038262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc6af4-313f-4a68-845d-cd8d2c71e002 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:55.038734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc6af4-313f-4a68-845d-cd8d2c71e002 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:55.039222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dbc6af4-313f-4a68-845d-cd8d2c71e002 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:55.040085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc6af4-313f-4a68-845d-cd8d2c71e002 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:55.041649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dbc6af4-313f-4a68-845d-cd8d2c71e002 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:55.063639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 469/918] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:48:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:55.301825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979fbb5-bbb0-43ff-b6af-638f58cf9dd4 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:55.302201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979fbb5-bbb0-43ff-b6af-638f58cf9dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:55.302322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1979fbb5-bbb0-43ff-b6af-638f58cf9dd4 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:55.303910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 450/919] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:55.310263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c754f45-4e16-4dd0-8c21-ab0282f94919 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:55.310546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c754f45-4e16-4dd0-8c21-ab0282f94919 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:55.310822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c754f45-4e16-4dd0-8c21-ab0282f94919 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:55.331342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7c754f45-4e16-4dd0-8c21-ab0282f94919 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:55.332862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7c754f45-4e16-4dd0-8c21-ab0282f94919 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:55.351577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:55.413544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 470/920] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:55.676289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979fbb5-bbb0-43ff-b6af-638f58cf9dd4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:55.694165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:55.785679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1979fbb5-bbb0-43ff-b6af-638f58cf9dd4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:55.796952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d30de-8cbe-4532-89b7-91fe9dc4ffc9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:55.797903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d30de-8cbe-4532-89b7-91fe9dc4ffc9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:55.798408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277d30de-8cbe-4532-89b7-91fe9dc4ffc9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:55.803985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 471/921] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:55.807695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dca40-323d-473e-b590-da6ca78b1948 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:55.808627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dca40-323d-473e-b590-da6ca78b1948 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:55.809099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01dca40-323d-473e-b590-da6ca78b1948 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:55.821560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dda61-cb5d-4baa-9390-fc461061b23d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:55.822240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dda61-cb5d-4baa-9390-fc461061b23d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:55.822525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dda61-cb5d-4baa-9390-fc461061b23d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:55.822803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8dda61-cb5d-4baa-9390-fc461061b23d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:55.823445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8dda61-cb5d-4baa-9390-fc461061b23d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:55.825142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8dda61-cb5d-4baa-9390-fc461061b23d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:55.831587 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b01dca40-323d-473e-b590-da6ca78b1948 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:55.833287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b01dca40-323d-473e-b590-da6ca78b1948 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-3f8dda61-cb5d-4baa-9390-fc461061b23d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:55.834439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:55.835463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:55.838951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 451/922] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:48:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:55.842237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dda61-cb5d-4baa-9390-fc461061b23d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:55.927739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:55.937618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 472/923] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:55 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:55.953982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8dda61-cb5d-4baa-9390-fc461061b23d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:55.989458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0343e3-a6f8-4f51-92df-bbb1cfdae7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:55.990343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0343e3-a6f8-4f51-92df-bbb1cfdae7ae None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:55.990786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0343e3-a6f8-4f51-92df-bbb1cfdae7ae None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:55.991195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0343e3-a6f8-4f51-92df-bbb1cfdae7ae None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:55.991957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0343e3-a6f8-4f51-92df-bbb1cfdae7ae None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:55.992495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0343e3-a6f8-4f51-92df-bbb1cfdae7ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:55.994002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0343e3-a6f8-4f51-92df-bbb1cfdae7ae None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:55.999816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 452/924] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:48:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:56.007241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dca40-323d-473e-b590-da6ca78b1948 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.104501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:56.125823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01dca40-323d-473e-b590-da6ca78b1948 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:56.161424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad49a32-8cfc-4bc6-a36f-731c98e2dd35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:56.162308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49a32-8cfc-4bc6-a36f-731c98e2dd35 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.162727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49a32-8cfc-4bc6-a36f-731c98e2dd35 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.163137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad49a32-8cfc-4bc6-a36f-731c98e2dd35 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.163929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad49a32-8cfc-4bc6-a36f-731c98e2dd35 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:56.164366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad49a32-8cfc-4bc6-a36f-731c98e2dd35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:56.165880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad49a32-8cfc-4bc6-a36f-731c98e2dd35 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:56.168098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a96158-5bcd-4037-9c54-45e026c0b02c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.168375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a96158-5bcd-4037-9c54-45e026c0b02c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.168645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a96158-5bcd-4037-9c54-45e026c0b02c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.170917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 453/925] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:48:56.173591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 473/926] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:48:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:56.179582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b560d9-4307-4187-b0fc-7c0dc7e6a3c9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.179912 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b560d9-4307-4187-b0fc-7c0dc7e6a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.180244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b560d9-4307-4187-b0fc-7c0dc7e6a3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.181262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad49a32-8cfc-4bc6-a36f-731c98e2dd35 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.205071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-37b560d9-4307-4187-b0fc-7c0dc7e6a3c9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:56.206854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-37b560d9-4307-4187-b0fc-7c0dc7e6a3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:56.281061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad49a32-8cfc-4bc6-a36f-731c98e2dd35 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:56.294452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 454/927] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:56 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:48:56.294452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b560d9-4307-4187-b0fc-7c0dc7e6a3c9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.321218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94677cb0-9d6c-4d32-97ab-56c7475a6670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:56.322228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94677cb0-9d6c-4d32-97ab-56c7475a6670 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.322642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94677cb0-9d6c-4d32-97ab-56c7475a6670 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.323057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94677cb0-9d6c-4d32-97ab-56c7475a6670 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.324063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94677cb0-9d6c-4d32-97ab-56c7475a6670 None admin] RBAC: Adding query filter params (interface=public, service_id=cbf6f05b78f7429f993f8c5e27f172fa, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:56.324594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94677cb0-9d6c-4d32-97ab-56c7475a6670 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:56.326175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94677cb0-9d6c-4d32-97ab-56c7475a6670 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:56.336524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 474/928] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Sep 13 06:48:56 2022] GET /identity/v3/endpoints?service_id=cbf6f05b78f7429f993f8c5e27f172fa&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:56.337683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94677cb0-9d6c-4d32-97ab-56c7475a6670 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.348659 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:56.372518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:56.407799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b560d9-4307-4187-b0fc-7c0dc7e6a3c9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:56.445424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94677cb0-9d6c-4d32-97ab-56c7475a6670 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:56.446568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c6757-37e5-4725-b1b1-7e703fe05b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:56.447206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c6757-37e5-4725-b1b1-7e703fe05b8c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.447519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c6757-37e5-4725-b1b1-7e703fe05b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.447798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767c6757-37e5-4725-b1b1-7e703fe05b8c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.448389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c6757-37e5-4725-b1b1-7e703fe05b8c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:56.449962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767c6757-37e5-4725-b1b1-7e703fe05b8c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-767c6757-37e5-4725-b1b1-7e703fe05b8c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:56.458914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:56.460024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:56.460911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 455/929] 10.222.0.25 () {56 vars in 1132 bytes} [Tue Sep 13 06:48:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:56.466290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c6757-37e5-4725-b1b1-7e703fe05b8c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.486243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d967b-5c21-4a5e-8ec7-ea1130ed19bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:56.487187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d967b-5c21-4a5e-8ec7-ea1130ed19bc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.487685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d967b-5c21-4a5e-8ec7-ea1130ed19bc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.488145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137d967b-5c21-4a5e-8ec7-ea1130ed19bc None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.488933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d967b-5c21-4a5e-8ec7-ea1130ed19bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:56.490627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137d967b-5c21-4a5e-8ec7-ea1130ed19bc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:56.497259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 475/930] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Sep 13 06:48:56 2022] GET /identity/v3/services => generated 1497 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 06:48:56.571348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d967b-5c21-4a5e-8ec7-ea1130ed19bc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.578188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767c6757-37e5-4725-b1b1-7e703fe05b8c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:56.622832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aed13b-0b62-4bab-b9cb-9123123f10ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:56.623466 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aed13b-0b62-4bab-b9cb-9123123f10ea None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.623746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aed13b-0b62-4bab-b9cb-9123123f10ea None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.624020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4aed13b-0b62-4bab-b9cb-9123123f10ea None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.624713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aed13b-0b62-4bab-b9cb-9123123f10ea None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:56.625023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aed13b-0b62-4bab-b9cb-9123123f10ea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:56.626499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4aed13b-0b62-4bab-b9cb-9123123f10ea None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:56.628666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:56.637558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 456/931] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:48:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:56.639422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aed13b-0b62-4bab-b9cb-9123123f10ea None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.677624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:56.712398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137d967b-5c21-4a5e-8ec7-ea1130ed19bc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:56.753949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4aed13b-0b62-4bab-b9cb-9123123f10ea None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:56.754225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e446b-c063-4b20-8649-670d968f674e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:56.754757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e446b-c063-4b20-8649-670d968f674e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.755026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e446b-c063-4b20-8649-670d968f674e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.755442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e446b-c063-4b20-8649-670d968f674e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.766459 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e446b-c063-4b20-8649-670d968f674e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8e9e446b-c063-4b20-8649-670d968f674e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:48:56.771498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:56.772423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:56.773515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 476/932] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Sep 13 06:48:56 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:56.779740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e446b-c063-4b20-8649-670d968f674e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.798770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e8f93-17ca-4441-a5e2-8b0be1e71a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:56.799474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e8f93-17ca-4441-a5e2-8b0be1e71a51 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.799660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e8f93-17ca-4441-a5e2-8b0be1e71a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.799930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803e8f93-17ca-4441-a5e2-8b0be1e71a51 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.800562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e8f93-17ca-4441-a5e2-8b0be1e71a51 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:56.800862 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e8f93-17ca-4441-a5e2-8b0be1e71a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:56.802887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803e8f93-17ca-4441-a5e2-8b0be1e71a51 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:56.810114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 457/933] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:48:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:56.815678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e8f93-17ca-4441-a5e2-8b0be1e71a51 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.892109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e446b-c063-4b20-8649-670d968f674e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:56.900735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:56.926025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:56.932475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fd0f7-d548-4d76-912d-a202815a17c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:56.933946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fd0f7-d548-4d76-912d-a202815a17c9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.934323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fd0f7-d548-4d76-912d-a202815a17c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.934959 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64fd0f7-d548-4d76-912d-a202815a17c9 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.935489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fd0f7-d548-4d76-912d-a202815a17c9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:56.935870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fd0f7-d548-4d76-912d-a202815a17c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:56.937406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64fd0f7-d548-4d76-912d-a202815a17c9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:56.941437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803e8f93-17ca-4441-a5e2-8b0be1e71a51 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:56.946801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 477/934] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Sep 13 06:48:56 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:56.952745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fd0f7-d548-4d76-912d-a202815a17c9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:56.981695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510503c9-e3cf-4ee4-a2b8-30e5b316b131 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:56.982793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510503c9-e3cf-4ee4-a2b8-30e5b316b131 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:56.983075 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510503c9-e3cf-4ee4-a2b8-30e5b316b131 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:56.983346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510503c9-e3cf-4ee4-a2b8-30e5b316b131 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:56.984023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510503c9-e3cf-4ee4-a2b8-30e5b316b131 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:56.985415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510503c9-e3cf-4ee4-a2b8-30e5b316b131 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:57.021721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 458/935] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:48:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:48:57.070110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64fd0f7-d548-4d76-912d-a202815a17c9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:57.117388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dddb33-833f-436a-b592-32986b74c899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:57.118158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dddb33-833f-436a-b592-32986b74c899 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:57.118437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dddb33-833f-436a-b592-32986b74c899 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:57.118708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78dddb33-833f-436a-b592-32986b74c899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:57.131816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dddb33-833f-436a-b592-32986b74c899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:57.133429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78dddb33-833f-436a-b592-32986b74c899 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-78dddb33-833f-436a-b592-32986b74c899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:57.149962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:57.151104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:57.152029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:57.152029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:57.152029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:57.152029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 478/936] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:57.157630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510503c9-e3cf-4ee4-a2b8-30e5b316b131 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:57.270700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510503c9-e3cf-4ee4-a2b8-30e5b316b131 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:57.291279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:57.306155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120c91c1-ccc0-4d68-abfd-26e68dc69798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:57.306790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c91c1-ccc0-4d68-abfd-26e68dc69798 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:57.307243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c91c1-ccc0-4d68-abfd-26e68dc69798 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:57.307531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120c91c1-ccc0-4d68-abfd-26e68dc69798 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:57.308710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c91c1-ccc0-4d68-abfd-26e68dc69798 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:57.309022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c91c1-ccc0-4d68-abfd-26e68dc69798 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:57.310691 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120c91c1-ccc0-4d68-abfd-26e68dc69798 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:57.327888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 459/937] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:57.330481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289476d-7349-481c-b2cc-21761f18254e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:57.330764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289476d-7349-481c-b2cc-21761f18254e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:57.330938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dddb33-833f-436a-b592-32986b74c899 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:57.331092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289476d-7349-481c-b2cc-21761f18254e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:57.332572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 460/938] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:57.339703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29e7f2-c98a-48c0-95d9-286980f87ac1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:57.340001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29e7f2-c98a-48c0-95d9-286980f87ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:57.340259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef29e7f2-c98a-48c0-95d9-286980f87ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:57.359366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ef29e7f2-c98a-48c0-95d9-286980f87ac1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:57.360650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ef29e7f2-c98a-48c0-95d9-286980f87ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:57.383144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:57.431072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 461/939] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:57.461223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78dddb33-833f-436a-b592-32986b74c899 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:57.498235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787411e1-a250-4d33-858d-57d4ac39e23a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:57.498885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787411e1-a250-4d33-858d-57d4ac39e23a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:57.499166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787411e1-a250-4d33-858d-57d4ac39e23a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:57.499435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787411e1-a250-4d33-858d-57d4ac39e23a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:57.508434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787411e1-a250-4d33-858d-57d4ac39e23a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:57.510155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787411e1-a250-4d33-858d-57d4ac39e23a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-787411e1-a250-4d33-858d-57d4ac39e23a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:48:57.523989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:57.524992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:48:57.525954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:57.530213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 479/940] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:48:57 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:57.532135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29e7f2-c98a-48c0-95d9-286980f87ac1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:57.648499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef29e7f2-c98a-48c0-95d9-286980f87ac1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:57.660011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:57.685259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185b2a2-dc62-4e43-9db8-9fb30f454273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:57.685877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185b2a2-dc62-4e43-9db8-9fb30f454273 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:57.686191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185b2a2-dc62-4e43-9db8-9fb30f454273 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:57.686464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4185b2a2-dc62-4e43-9db8-9fb30f454273 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:57.687195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185b2a2-dc62-4e43-9db8-9fb30f454273 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:57.687499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185b2a2-dc62-4e43-9db8-9fb30f454273 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:57.688935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4185b2a2-dc62-4e43-9db8-9fb30f454273 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:57.704617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787411e1-a250-4d33-858d-57d4ac39e23a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:57.714195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 462/941] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Sep 13 06:48:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:57.718539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185b2a2-dc62-4e43-9db8-9fb30f454273 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:57.832224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4185b2a2-dc62-4e43-9db8-9fb30f454273 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:57.837404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787411e1-a250-4d33-858d-57d4ac39e23a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:57.839223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:57.872616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6385d-92c6-4162-8245-88b8c01951ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:57.873507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6385d-92c6-4162-8245-88b8c01951ae None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:57.873924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6385d-92c6-4162-8245-88b8c01951ae None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:57.874373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6385d-92c6-4162-8245-88b8c01951ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:57.878426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:57.879305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:57.879783 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:57.880252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:57.881103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:57.883635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:57.885229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6385d-92c6-4162-8245-88b8c01951ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:57.886825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6385d-92c6-4162-8245-88b8c01951ae None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:57.887570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WARNING keystone.server.flask.common [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-59f6385d-92c6-4162-8245-88b8c01951ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:48:57.904191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:57.905230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:48:57.906211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:48:57.906211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:48:57.906211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:57.908373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 463/942] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:48:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:57.910496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.sql.core [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 06:48:57.910496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 06:48:57.910496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [parameters: {'id': 'cf0f8a4a9ecd44e5868dd15c6ca3809f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 06:48:57.910496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 06:48:57.915025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6385d-92c6-4162-8245-88b8c01951ae None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cf0f8a4a9ecd44e5868dd15c6ca3809f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cf0f8a4a9ecd44e5868dd15c6ca3809f. Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 06:48:57.918721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 13 06:48:57.919568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application yield resource Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise self.value Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application session.commit() Sep 13 06:48:57.920527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.flush() Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application compat.raise_( Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 06:48:57.921418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 06:48:57.922493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 06:48:57.923491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [parameters: {'id': 'cf0f8a4a9ecd44e5868dd15c6ca3809f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:57.924625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:57.925612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:57.926829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:57.926829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 13 06:48:57.926829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 13 06:48:57.926829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:57.926829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:57.926829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 13 06:48:57.926829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 13 06:48:57.926829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cf0f8a4a9ecd44e5868dd15c6ca3809f. Sep 13 06:48:57.926829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:57.929075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 480/943] 10.222.0.25 () {60 vars in 1140 bytes} [Tue Sep 13 06:48:57 2022] POST /identity/v3/projects => generated 262 bytes in 225 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:57.937097 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:57.945098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:58.034946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6385d-92c6-4162-8245-88b8c01951ae None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:58.056590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f4782b-a3b1-4055-a024-a5b83f89685d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:58.076093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9e5b1-50a1-4176-9d60-cfbb6ae0822f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:58.076868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9e5b1-50a1-4176-9d60-cfbb6ae0822f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:58.077242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9e5b1-50a1-4176-9d60-cfbb6ae0822f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:58.077616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a9e5b1-50a1-4176-9d60-cfbb6ae0822f None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:58.078396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9e5b1-50a1-4176-9d60-cfbb6ae0822f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:58.078793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9e5b1-50a1-4176-9d60-cfbb6ae0822f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:58.080682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a9e5b1-50a1-4176-9d60-cfbb6ae0822f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:58.094384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c64602-1329-4021-aeb0-642ce87f898a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:58.095344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c64602-1329-4021-aeb0-642ce87f898a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:58.095825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c64602-1329-4021-aeb0-642ce87f898a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:58.096294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c64602-1329-4021-aeb0-642ce87f898a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:58.099058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 464/944] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:48:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:58.100733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9e5b1-50a1-4176-9d60-cfbb6ae0822f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:58.104622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c64602-1329-4021-aeb0-642ce87f898a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:58.106594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c64602-1329-4021-aeb0-642ce87f898a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e5c64602-1329-4021-aeb0-642ce87f898a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:58.119092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:58.119995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:58.120992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:58.120992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:58.120992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:58.120992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:58.120992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:58.120992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:58.120992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:58.120992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:58.128194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 481/945] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:58.129384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c64602-1329-4021-aeb0-642ce87f898a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:58.150053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:58.191839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:58.213783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a9e5b1-50a1-4176-9d60-cfbb6ae0822f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:58.243309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c64602-1329-4021-aeb0-642ce87f898a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:58.252733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b45aa-24af-4db2-bff6-a6d3ed3053bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:58.253378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b45aa-24af-4db2-bff6-a6d3ed3053bf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:58.253655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b45aa-24af-4db2-bff6-a6d3ed3053bf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:58.253927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88b45aa-24af-4db2-bff6-a6d3ed3053bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:58.261860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b45aa-24af-4db2-bff6-a6d3ed3053bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:58.263498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88b45aa-24af-4db2-bff6-a6d3ed3053bf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-b88b45aa-24af-4db2-bff6-a6d3ed3053bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:48:58.276367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:58.277399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:58.278352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:48:58.278352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:48:58.278352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:48:58.278352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:48:58.278352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:48:58.278352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:48:58.278352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:48:58.278352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:58.279334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 465/946] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:58.284375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b45aa-24af-4db2-bff6-a6d3ed3053bf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:58.286688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675b383-3854-442d-948c-274c18f49cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:58.287319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675b383-3854-442d-948c-274c18f49cce None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:58.287788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675b383-3854-442d-948c-274c18f49cce None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:58.287975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b675b383-3854-442d-948c-274c18f49cce None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:58.288656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675b383-3854-442d-948c-274c18f49cce None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:58.288962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675b383-3854-442d-948c-274c18f49cce None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:58.290556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b675b383-3854-442d-948c-274c18f49cce None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:58.322452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 482/947] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:48:58 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:58.400321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88b45aa-24af-4db2-bff6-a6d3ed3053bf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:58.438801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf68b72-66e6-4e07-82bd-a1060072b123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:58.439902 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf68b72-66e6-4e07-82bd-a1060072b123 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:58.440380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf68b72-66e6-4e07-82bd-a1060072b123 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:58.440857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf68b72-66e6-4e07-82bd-a1060072b123 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:58.441771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf68b72-66e6-4e07-82bd-a1060072b123 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:58.442319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf68b72-66e6-4e07-82bd-a1060072b123 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:58.443981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf68b72-66e6-4e07-82bd-a1060072b123 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:58.455604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:58.464103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 466/948] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:48:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:58.467233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675b383-3854-442d-948c-274c18f49cce None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:58.550565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:58.573711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b675b383-3854-442d-948c-274c18f49cce None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:58.618167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb05bb3e-1292-4e77-86c6-36c66fcf4e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:58.619067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05bb3e-1292-4e77-86c6-36c66fcf4e57 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:58.619489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05bb3e-1292-4e77-86c6-36c66fcf4e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:58.619904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb05bb3e-1292-4e77-86c6-36c66fcf4e57 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/7363637f44264a65b66ba53da825eaff/roles/d0cb30355e064a3d8562073f93479915` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:58.622610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05bb3e-1292-4e77-86c6-36c66fcf4e57 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=7363637f44264a65b66ba53da825eaff, role_id=d0cb30355e064a3d8562073f93479915)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:58.624220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb05bb3e-1292-4e77-86c6-36c66fcf4e57 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:58.652446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 483/949] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:48:58 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/7363637f44264a65b66ba53da825eaff/roles/d0cb30355e064a3d8562073f93479915 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:48:58.853652 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa354b-7ac7-453f-b2d1-24c082a7214a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:58.854792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa354b-7ac7-453f-b2d1-24c082a7214a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:58.855400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aa354b-7ac7-453f-b2d1-24c082a7214a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:58.857368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 467/950] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:58.862894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb99c8-04a8-4abb-bc2c-124c4df51aa8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:58.863434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb99c8-04a8-4abb-bc2c-124c4df51aa8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:58.863903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45cb99c8-04a8-4abb-bc2c-124c4df51aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:58.883790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-45cb99c8-04a8-4abb-bc2c-124c4df51aa8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:58.885723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-45cb99c8-04a8-4abb-bc2c-124c4df51aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:58.958692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 484/951] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:48:59.221833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa354b-7ac7-453f-b2d1-24c082a7214a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:59.244556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7068e2b-0856-410a-a1be-7f44589d8bb9 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:59.244895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7068e2b-0856-410a-a1be-7f44589d8bb9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:59.245108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7068e2b-0856-410a-a1be-7f44589d8bb9 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:59.246998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 485/952] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:48:59.252169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ace60-867b-4c01-b6ec-9faf64e8bfa5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:59.252439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ace60-867b-4c01-b6ec-9faf64e8bfa5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:59.252718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57ace60-867b-4c01-b6ec-9faf64e8bfa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:59.270971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a57ace60-867b-4c01-b6ec-9faf64e8bfa5 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:48:59.272164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a57ace60-867b-4c01-b6ec-9faf64e8bfa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:48:59.332435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47aa354b-7ac7-453f-b2d1-24c082a7214a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:59.358968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:59.374105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 486/953] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:48:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:48:59.378200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf011f-0a08-465b-a344-be3622fefd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:59.378881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf011f-0a08-465b-a344-be3622fefd01 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:59.379157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf011f-0a08-465b-a344-be3622fefd01 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:59.379687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52bf011f-0a08-465b-a344-be3622fefd01 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:59.380187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf011f-0a08-465b-a344-be3622fefd01 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:59.381565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52bf011f-0a08-465b-a344-be3622fefd01 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-52bf011f-0a08-465b-a344-be3622fefd01 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:59.390757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:59.392007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:48:59.392892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 468/954] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:48:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:48:59.397867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ace60-867b-4c01-b6ec-9faf64e8bfa5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:59.507688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57ace60-867b-4c01-b6ec-9faf64e8bfa5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:59.541902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe7392-aa54-4c85-a4a6-af15823b9ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:59.542621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe7392-aa54-4c85-a4a6-af15823b9ca6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:59.542907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe7392-aa54-4c85-a4a6-af15823b9ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:59.543177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ebe7392-aa54-4c85-a4a6-af15823b9ca6 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:59.543808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe7392-aa54-4c85-a4a6-af15823b9ca6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:59.544112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe7392-aa54-4c85-a4a6-af15823b9ca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:59.545564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ebe7392-aa54-4c85-a4a6-af15823b9ca6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:59.550902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 487/955] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:48:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:48:59.556757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf011f-0a08-465b-a344-be3622fefd01 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:59.636931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:59.651419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe7392-aa54-4c85-a4a6-af15823b9ca6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:59.679589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52bf011f-0a08-465b-a344-be3622fefd01 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:59.718183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:59.724732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d00b7-b3d1-437d-b742-ec5c03fa9f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:59.725447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d00b7-b3d1-437d-b742-ec5c03fa9f26 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:59.725725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d00b7-b3d1-437d-b742-ec5c03fa9f26 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:59.725995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d00b7-b3d1-437d-b742-ec5c03fa9f26 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:59.726846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d00b7-b3d1-437d-b742-ec5c03fa9f26 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:48:59.727146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d00b7-b3d1-437d-b742-ec5c03fa9f26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:59.728519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4d00b7-b3d1-437d-b742-ec5c03fa9f26 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:59.735400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 469/956] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:48:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:48:59.741074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d00b7-b3d1-437d-b742-ec5c03fa9f26 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:59.762297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ebe7392-aa54-4c85-a4a6-af15823b9ca6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:59.805465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96ec94-3148-40c0-babd-370ff7e302e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:59.806462 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96ec94-3148-40c0-babd-370ff7e302e2 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:59.806800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96ec94-3148-40c0-babd-370ff7e302e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:59.807131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f96ec94-3148-40c0-babd-370ff7e302e2 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:59.808132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96ec94-3148-40c0-babd-370ff7e302e2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:59.809778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f96ec94-3148-40c0-babd-370ff7e302e2 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2f96ec94-3148-40c0-babd-370ff7e302e2 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:48:59.828199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:48:59.829278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:48:59.830208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 488/957] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:48:59 2022] GET /identity/v3/services/compute => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:48:59.843008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96ec94-3148-40c0-babd-370ff7e302e2 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:48:59.846920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4d00b7-b3d1-437d-b742-ec5c03fa9f26 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:59.882607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:48:59.883656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a42f5d-88dc-4705-9092-2b45d893fb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:48:59.884386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a42f5d-88dc-4705-9092-2b45d893fb4e None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:48:59.884730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a42f5d-88dc-4705-9092-2b45d893fb4e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:48:59.885124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a42f5d-88dc-4705-9092-2b45d893fb4e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:48:59.886022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a42f5d-88dc-4705-9092-2b45d893fb4e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:48:59.887712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a42f5d-88dc-4705-9092-2b45d893fb4e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:48:59.918468 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 470/958] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:48:59 2022] POST /identity/v3/endpoints => generated 349 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:48:59.952066 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f96ec94-3148-40c0-babd-370ff7e302e2 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:48:59.981290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:00.001918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ef7dfa-0fc2-4849-bf80-42da849f88fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:00.002657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef7dfa-0fc2-4849-bf80-42da849f88fc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:00.002935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef7dfa-0fc2-4849-bf80-42da849f88fc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:00.003206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ef7dfa-0fc2-4849-bf80-42da849f88fc None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:00.003919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef7dfa-0fc2-4849-bf80-42da849f88fc None admin] RBAC: Adding query filter params (name=compute) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:00.004225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef7dfa-0fc2-4849-bf80-42da849f88fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:00.005699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ef7dfa-0fc2-4849-bf80-42da849f88fc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:00.011925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 489/959] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:48:59 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:00.017709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a42f5d-88dc-4705-9092-2b45d893fb4e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:00.135051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a42f5d-88dc-4705-9092-2b45d893fb4e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:00.171765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4156b831-03c8-4fb7-be8a-2295206e491c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:00.172445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156b831-03c8-4fb7-be8a-2295206e491c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:00.172730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156b831-03c8-4fb7-be8a-2295206e491c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:00.173004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4156b831-03c8-4fb7-be8a-2295206e491c None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:00.173639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156b831-03c8-4fb7-be8a-2295206e491c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:00.173941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156b831-03c8-4fb7-be8a-2295206e491c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:00.175363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4156b831-03c8-4fb7-be8a-2295206e491c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:00.181380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 471/960] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:49:00 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:00.691092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21216a4-a4ab-4f00-a2e3-8a7db737265c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:00.691437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21216a4-a4ab-4f00-a2e3-8a7db737265c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:00.691644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21216a4-a4ab-4f00-a2e3-8a7db737265c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:00.693302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 490/961] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:00.698847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e534665-aded-49fc-8a4f-6c2a151f106f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:00.699188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e534665-aded-49fc-8a4f-6c2a151f106f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:00.699353 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e534665-aded-49fc-8a4f-6c2a151f106f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:00.718815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5e534665-aded-49fc-8a4f-6c2a151f106f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:00.720019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e534665-aded-49fc-8a4f-6c2a151f106f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:00.769606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:00.800343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 472/962] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:00.902355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8606d72-37d7-46be-b57a-97aed0a7d534 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:00.902536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8606d72-37d7-46be-b57a-97aed0a7d534 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:00.902805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8606d72-37d7-46be-b57a-97aed0a7d534 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:00.904492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 491/963] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:00.910727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952d9e4-225e-4946-8915-56951e80e09c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:00.911226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952d9e4-225e-4946-8915-56951e80e09c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:00.911304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952d9e4-225e-4946-8915-56951e80e09c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:00.932582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9952d9e4-225e-4946-8915-56951e80e09c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:00.934275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9952d9e4-225e-4946-8915-56951e80e09c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:01.014801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 473/964] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:00 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:01.118098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8606d72-37d7-46be-b57a-97aed0a7d534 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:01.239622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8606d72-37d7-46be-b57a-97aed0a7d534 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:01.281207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:01.281835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:01.282133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:01.282413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:01.283438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:01.285290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:01.304241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.sql.core [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 06:49:01.304241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 06:49:01.304241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [parameters: {'user_id': 'bb080e4d5a7d44229a24008ce452ab84', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 06:49:01.304241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81500) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 13 06:49:01.312751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 06:49:01.314391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application yield resource Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application next(self.gen) Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.rollback() Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.force_reraise() Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise self.value Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application session.commit() Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 13 06:49:01.316862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._prepare_impl() Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.session.flush() Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._flush(objects) Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application compat.raise_( Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application flush_context.execute() Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rec.execute(self) Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 13 06:49:01.318508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 13 06:49:01.319619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result = self._query(query) Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application conn.query(q) Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application result.read() Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 13 06:49:01.320750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [parameters: {'user_id': 'bb080e4d5a7d44229a24008ce452ab84', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:01.321809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 13 06:49:01.322846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 13 06:49:01.324040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 13 06:49:01.324040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 13 06:49:01.324040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 13 06:49:01.324040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 13 06:49:01.324040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 13 06:49:01.324040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:01.324654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952d9e4-225e-4946-8915-56951e80e09c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:01.324744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 492/965] 10.222.0.25 () {60 vars in 1131 bytes} [Tue Sep 13 06:49:01 2022] POST /identity/v3/users => generated 160 bytes in 197 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:01.324744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:01.402270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:01.442379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9952d9e4-225e-4946-8915-56951e80e09c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:01.458467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:01.470117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6c0e52-41eb-4baf-b05e-662b7a763f3c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:01.488102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaadad9-30f6-4dea-a972-6dd2911bc622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:01.489027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaadad9-30f6-4dea-a972-6dd2911bc622 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:01.489308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaadad9-30f6-4dea-a972-6dd2911bc622 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:01.489581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaadad9-30f6-4dea-a972-6dd2911bc622 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:01.503081 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaadad9-30f6-4dea-a972-6dd2911bc622 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:01.509466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbbefc-9cda-4ff2-aa53-6e2542f91b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:01.510266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbbefc-9cda-4ff2-aa53-6e2542f91b8c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-efaadad9-30f6-4dea-a972-6dd2911bc622 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:01.510398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:01.511519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:01.512437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 474/966] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Sep 13 06:49:01 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:01.516949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbbefc-9cda-4ff2-aa53-6e2542f91b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:01.517342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cbbefc-9cda-4ff2-aa53-6e2542f91b8c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:01.518299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaadad9-30f6-4dea-a972-6dd2911bc622 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:01.527190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbbefc-9cda-4ff2-aa53-6e2542f91b8c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:01.528965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cbbefc-9cda-4ff2-aa53-6e2542f91b8c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e0cbbefc-9cda-4ff2-aa53-6e2542f91b8c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:49:01.541720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:01.544029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:01.546477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 493/967] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:49:01 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:01.555309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbbefc-9cda-4ff2-aa53-6e2542f91b8c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:01.673050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaadad9-30f6-4dea-a972-6dd2911bc622 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:01.700546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cbbefc-9cda-4ff2-aa53-6e2542f91b8c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:01.713001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:01.740519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43b6ed-6886-40c0-8406-cb1d1cd9ec3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:01.741694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43b6ed-6886-40c0-8406-cb1d1cd9ec3b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:01.742185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43b6ed-6886-40c0-8406-cb1d1cd9ec3b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:01.742922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da43b6ed-6886-40c0-8406-cb1d1cd9ec3b None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:01.744156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43b6ed-6886-40c0-8406-cb1d1cd9ec3b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:01.744875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43b6ed-6886-40c0-8406-cb1d1cd9ec3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:01.747770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da43b6ed-6886-40c0-8406-cb1d1cd9ec3b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:01.752136 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a91b7-b334-4073-a732-e2acf62415cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:01.753187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a91b7-b334-4073-a732-e2acf62415cf None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:01.753486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a91b7-b334-4073-a732-e2acf62415cf None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:01.753876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308a91b7-b334-4073-a732-e2acf62415cf None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:01.754723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a91b7-b334-4073-a732-e2acf62415cf None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:01.755145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a91b7-b334-4073-a732-e2acf62415cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:01.756774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308a91b7-b334-4073-a732-e2acf62415cf None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:01.761471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 475/968] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Sep 13 06:49:01 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:01.770207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43b6ed-6886-40c0-8406-cb1d1cd9ec3b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:01.804655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 494/969] 10.222.0.25 () {56 vars in 1101 bytes} [Tue Sep 13 06:49:01 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:01.812890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:01.903343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da43b6ed-6886-40c0-8406-cb1d1cd9ec3b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:01.949205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c009217-1b1d-472d-ac7a-7bc27a921302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:01.949887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c009217-1b1d-472d-ac7a-7bc27a921302 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:01.950165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c009217-1b1d-472d-ac7a-7bc27a921302 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:01.950447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c009217-1b1d-472d-ac7a-7bc27a921302 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:01.967885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c009217-1b1d-472d-ac7a-7bc27a921302 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:01.969390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c009217-1b1d-472d-ac7a-7bc27a921302 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6c009217-1b1d-472d-ac7a-7bc27a921302 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:01.988738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:01.989794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:49:01.990763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:01.990763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:01.990763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:01.990763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 476/970] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:01.997697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a91b7-b334-4073-a732-e2acf62415cf None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:02.117739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308a91b7-b334-4073-a732-e2acf62415cf None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:02.153111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3ef19-9184-4d4b-9c16-ee6580be5bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:02.153784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3ef19-9184-4d4b-9c16-ee6580be5bec None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:02.154025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3ef19-9184-4d4b-9c16-ee6580be5bec None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:02.154303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f3ef19-9184-4d4b-9c16-ee6580be5bec None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:02.154963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3ef19-9184-4d4b-9c16-ee6580be5bec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:02.155269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3ef19-9184-4d4b-9c16-ee6580be5bec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:02.156733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f3ef19-9184-4d4b-9c16-ee6580be5bec None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:02.169405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 495/971] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:49:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:02.175430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c009217-1b1d-472d-ac7a-7bc27a921302 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:02.265148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:02.287733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c009217-1b1d-472d-ac7a-7bc27a921302 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:02.323338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db312ae4-5331-4033-858b-97f2b92028e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:02.324060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312ae4-5331-4033-858b-97f2b92028e9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:02.324314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312ae4-5331-4033-858b-97f2b92028e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:02.324596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db312ae4-5331-4033-858b-97f2b92028e9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:02.332422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db312ae4-5331-4033-858b-97f2b92028e9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:02.333865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db312ae4-5331-4033-858b-97f2b92028e9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-db312ae4-5331-4033-858b-97f2b92028e9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:49:02.345509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:02.346740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 13 06:49:02.347694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:02.350181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 477/972] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:49:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:02.352538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3ef19-9184-4d4b-9c16-ee6580be5bec None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:02.374876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4449c9-754a-4044-8dd4-86b3dd87618d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:02.375061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4449c9-754a-4044-8dd4-86b3dd87618d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:02.375592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4449c9-754a-4044-8dd4-86b3dd87618d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:02.376876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 478/973] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:49:02.382947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d560d7e-2838-4079-9fb1-8dc61dbcf7da None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:02.383268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d560d7e-2838-4079-9fb1-8dc61dbcf7da None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:02.383499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d560d7e-2838-4079-9fb1-8dc61dbcf7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:02.387081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:02.403881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3d560d7e-2838-4079-9fb1-8dc61dbcf7da None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:02.405108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3d560d7e-2838-4079-9fb1-8dc61dbcf7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:02.472943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f3ef19-9184-4d4b-9c16-ee6580be5bec None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:02.474967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 479/974] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:02.496702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6430785-1dc5-495e-ae11-6456c7d0d6b2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:02.496999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6430785-1dc5-495e-ae11-6456c7d0d6b2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:02.497271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6430785-1dc5-495e-ae11-6456c7d0d6b2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:02.498827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 480/975] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:49:02.503664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdaac0-c83d-4d9e-b680-dfdcb6d98ebc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:02.503979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdaac0-c83d-4d9e-b680-dfdcb6d98ebc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:02.504236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bdaac0-c83d-4d9e-b680-dfdcb6d98ebc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:02.509779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5066bdd-a26f-4f98-a792-affaf878ee83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:02.510411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5066bdd-a26f-4f98-a792-affaf878ee83 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:02.510708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5066bdd-a26f-4f98-a792-affaf878ee83 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:02.511067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5066bdd-a26f-4f98-a792-affaf878ee83 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:02.511792 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5066bdd-a26f-4f98-a792-affaf878ee83 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:02.512100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5066bdd-a26f-4f98-a792-affaf878ee83 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:02.513541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5066bdd-a26f-4f98-a792-affaf878ee83 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:02.522794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-45bdaac0-c83d-4d9e-b680-dfdcb6d98ebc None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:02.524162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-45bdaac0-c83d-4d9e-b680-dfdcb6d98ebc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:02.531343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 496/976] 10.222.0.25 () {56 vars in 1149 bytes} [Tue Sep 13 06:49:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:02.536995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5066bdd-a26f-4f98-a792-affaf878ee83 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:02.602777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 481/977] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:02.632359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:02.649237 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5066bdd-a26f-4f98-a792-affaf878ee83 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:02.683987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21454d-dea9-42ba-b3cf-9b946fb607a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:02.684603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21454d-dea9-42ba-b3cf-9b946fb607a7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:02.684885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21454d-dea9-42ba-b3cf-9b946fb607a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:02.685158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed21454d-dea9-42ba-b3cf-9b946fb607a7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:02.737244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdaac0-c83d-4d9e-b680-dfdcb6d98ebc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:02.740424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:02.753131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21454d-dea9-42ba-b3cf-9b946fb607a7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:02.754755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed21454d-dea9-42ba-b3cf-9b946fb607a7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:02.840910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:02.849036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bdaac0-c83d-4d9e-b680-dfdcb6d98ebc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:02.859189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 497/978] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:02 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:49:02.860365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21454d-dea9-42ba-b3cf-9b946fb607a7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:02.887794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e11561-ed3b-47cb-98ac-286fc7f03155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:02.888360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e11561-ed3b-47cb-98ac-286fc7f03155 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:02.888643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e11561-ed3b-47cb-98ac-286fc7f03155 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:02.888917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e11561-ed3b-47cb-98ac-286fc7f03155 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:02.889512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e11561-ed3b-47cb-98ac-286fc7f03155 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:02.890925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e11561-ed3b-47cb-98ac-286fc7f03155 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f7e11561-ed3b-47cb-98ac-286fc7f03155 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:02.900013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:02.901054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:02.902243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 482/979] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:49:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:02.903067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e11561-ed3b-47cb-98ac-286fc7f03155 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:02.969425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:02.971137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed21454d-dea9-42ba-b3cf-9b946fb607a7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.005571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0defa-55a2-4ec8-875e-7dffdad0e4d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.006215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0defa-55a2-4ec8-875e-7dffdad0e4d7 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.006390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e11561-ed3b-47cb-98ac-286fc7f03155 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.006493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0defa-55a2-4ec8-875e-7dffdad0e4d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.006746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa0defa-55a2-4ec8-875e-7dffdad0e4d7 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.007389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0defa-55a2-4ec8-875e-7dffdad0e4d7 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.008770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa0defa-55a2-4ec8-875e-7dffdad0e4d7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.024429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 498/980] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:49:02 2022] POST /identity/v3/services => generated 242 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:49:03.025289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0defa-55a2-4ec8-875e-7dffdad0e4d7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:03.054619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7997e-163a-413e-840c-8fb52655a5ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.055282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7997e-163a-413e-840c-8fb52655a5ac None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.055566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7997e-163a-413e-840c-8fb52655a5ac None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.055840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c7997e-163a-413e-840c-8fb52655a5ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.068355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7997e-163a-413e-840c-8fb52655a5ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.069852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c7997e-163a-413e-840c-8fb52655a5ac None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.081435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-96c7997e-163a-413e-840c-8fb52655a5ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:03.085332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:03.086605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:49:03.087748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:03.087748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:03.087748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:03.087748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 483/981] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:03.092885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7997e-163a-413e-840c-8fb52655a5ac None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:03.149384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa0defa-55a2-4ec8-875e-7dffdad0e4d7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.187198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4cd0e-d6ea-4d84-a357-45c36899e920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.187918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4cd0e-d6ea-4d84-a357-45c36899e920 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.188084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4cd0e-d6ea-4d84-a357-45c36899e920 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.188356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb4cd0e-d6ea-4d84-a357-45c36899e920 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.188993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4cd0e-d6ea-4d84-a357-45c36899e920 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:03.189300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4cd0e-d6ea-4d84-a357-45c36899e920 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.190731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb4cd0e-d6ea-4d84-a357-45c36899e920 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.197413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 499/982] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:49:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:03.204111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4cd0e-d6ea-4d84-a357-45c36899e920 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:03.218206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c7997e-163a-413e-840c-8fb52655a5ac None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.235454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:03.261173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa73802-7197-4959-8f0a-73c03e9b77b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.261794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73802-7197-4959-8f0a-73c03e9b77b0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.262070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73802-7197-4959-8f0a-73c03e9b77b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.262365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa73802-7197-4959-8f0a-73c03e9b77b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.263008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa73802-7197-4959-8f0a-73c03e9b77b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:03.263310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa73802-7197-4959-8f0a-73c03e9b77b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.265093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa73802-7197-4959-8f0a-73c03e9b77b0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.277394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 484/983] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:49:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:03.283425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa73802-7197-4959-8f0a-73c03e9b77b0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:03.321857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb4cd0e-d6ea-4d84-a357-45c36899e920 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.331753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:03.360115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21851429-d753-4724-a19b-0206510d266d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.360738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21851429-d753-4724-a19b-0206510d266d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.361020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21851429-d753-4724-a19b-0206510d266d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.361298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21851429-d753-4724-a19b-0206510d266d None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.361931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21851429-d753-4724-a19b-0206510d266d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:03.362272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21851429-d753-4724-a19b-0206510d266d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.363658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21851429-d753-4724-a19b-0206510d266d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.369193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 500/984] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:49:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:03.375164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21851429-d753-4724-a19b-0206510d266d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:03.390332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa73802-7197-4959-8f0a-73c03e9b77b0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.427203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dba3f7-d4e5-4daa-b7b1-5cade3ba83c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.427761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba3f7-d4e5-4daa-b7b1-5cade3ba83c7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.428042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba3f7-d4e5-4daa-b7b1-5cade3ba83c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.428316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dba3f7-d4e5-4daa-b7b1-5cade3ba83c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.437272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dba3f7-d4e5-4daa-b7b1-5cade3ba83c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.438783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37dba3f7-d4e5-4daa-b7b1-5cade3ba83c7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-37dba3f7-d4e5-4daa-b7b1-5cade3ba83c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:49:03.450755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:03.451680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:03.452589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:49:03.452589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:49:03.452589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:49:03.452589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:49:03.452589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:49:03.452589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:49:03.452589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:49:03.452589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:03.452589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 485/985] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:49:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:03.459094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dba3f7-d4e5-4daa-b7b1-5cade3ba83c7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:03.481360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21851429-d753-4724-a19b-0206510d266d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.495744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:03.515689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324e72b-431f-4fde-9d1a-fb93e014bc74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.516303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324e72b-431f-4fde-9d1a-fb93e014bc74 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.516584 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324e72b-431f-4fde-9d1a-fb93e014bc74 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.516857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6324e72b-431f-4fde-9d1a-fb93e014bc74 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.517562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324e72b-431f-4fde-9d1a-fb93e014bc74 None admin] RBAC: Adding query filter params (interface=admin, service_id=cbf6f05b78f7429f993f8c5e27f172fa, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:03.517873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324e72b-431f-4fde-9d1a-fb93e014bc74 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.519290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6324e72b-431f-4fde-9d1a-fb93e014bc74 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.524677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 501/986] 10.222.0.25 () {56 vars in 1249 bytes} [Tue Sep 13 06:49:03 2022] GET /identity/v3/endpoints?service_id=cbf6f05b78f7429f993f8c5e27f172fa&interface=admin®ion_id=RegionOne => generated 193 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:03.530802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324e72b-431f-4fde-9d1a-fb93e014bc74 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:03.567004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:03.576207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dba3f7-d4e5-4daa-b7b1-5cade3ba83c7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.620164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeada9f-4af8-4d8b-acca-8d06aa739be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.620738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeada9f-4af8-4d8b-acca-8d06aa739be0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.621003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeada9f-4af8-4d8b-acca-8d06aa739be0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.621274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeada9f-4af8-4d8b-acca-8d06aa739be0 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.622004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeada9f-4af8-4d8b-acca-8d06aa739be0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:03.622376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeada9f-4af8-4d8b-acca-8d06aa739be0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.623827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adeada9f-4af8-4d8b-acca-8d06aa739be0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.635735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 486/987] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:49:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:03.640225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6324e72b-431f-4fde-9d1a-fb93e014bc74 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.641786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeada9f-4af8-4d8b-acca-8d06aa739be0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:03.676573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c14797a-166e-4b16-bfda-b9a214aef7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.677185 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14797a-166e-4b16-bfda-b9a214aef7ac None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.677513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14797a-166e-4b16-bfda-b9a214aef7ac None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.677727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c14797a-166e-4b16-bfda-b9a214aef7ac None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.678463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14797a-166e-4b16-bfda-b9a214aef7ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.679862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c14797a-166e-4b16-bfda-b9a214aef7ac None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.687028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 502/988] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Sep 13 06:49:03 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 06:49:03.762202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:03.771264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeada9f-4af8-4d8b-acca-8d06aa739be0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:03.809438 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c9a11d-2b2a-4565-aa08-a05374d5bc46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:03.810071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9a11d-2b2a-4565-aa08-a05374d5bc46 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:03.810379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9a11d-2b2a-4565-aa08-a05374d5bc46 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:03.810656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c9a11d-2b2a-4565-aa08-a05374d5bc46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:03.811405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9a11d-2b2a-4565-aa08-a05374d5bc46 None admin] RBAC: Adding query filter params (role.id=d0cb30355e064a3d8562073f93479915, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=7363637f44264a65b66ba53da825eaff) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:03.811718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9a11d-2b2a-4565-aa08-a05374d5bc46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:03.813188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c9a11d-2b2a-4565-aa08-a05374d5bc46 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:03.820090 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 487/989] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:49:03 2022] GET /identity/v3/role_assignments?user.id=7363637f44264a65b66ba53da825eaff&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=d0cb30355e064a3d8562073f93479915 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:05.272190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47790555-a6a2-40e6-a070-ffc36c84700d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:05.272771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47790555-a6a2-40e6-a070-ffc36c84700d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:05.272771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47790555-a6a2-40e6-a070-ffc36c84700d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:05.274161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 503/990] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:05.280183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f156d-d4b1-4911-86dd-ffd3a053f3d3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:05.280611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f156d-d4b1-4911-86dd-ffd3a053f3d3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:05.280792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004f156d-d4b1-4911-86dd-ffd3a053f3d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:05.299845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-004f156d-d4b1-4911-86dd-ffd3a053f3d3 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:05.301039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-004f156d-d4b1-4911-86dd-ffd3a053f3d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:05.373187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 488/991] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:05.643952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47790555-a6a2-40e6-a070-ffc36c84700d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:05.750100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:05.755559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47790555-a6a2-40e6-a070-ffc36c84700d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:05.791079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29168bce-df0b-468f-88b1-70e258e78e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:05.791734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29168bce-df0b-468f-88b1-70e258e78e8a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:05.792009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29168bce-df0b-468f-88b1-70e258e78e8a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:05.792367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29168bce-df0b-468f-88b1-70e258e78e8a None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:05.793142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29168bce-df0b-468f-88b1-70e258e78e8a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:05.794631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29168bce-df0b-468f-88b1-70e258e78e8a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-29168bce-df0b-468f-88b1-70e258e78e8a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:05.804487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:05.805665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:05.811485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 504/992] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:49:05 2022] GET /identity/v3/services/compute => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:05.812660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f156d-d4b1-4911-86dd-ffd3a053f3d3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:05.909759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:05.911604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f5d26-8ea1-4d9b-b7dd-db24192a2b10 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:05.911902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f5d26-8ea1-4d9b-b7dd-db24192a2b10 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:05.912172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38f5d26-8ea1-4d9b-b7dd-db24192a2b10 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:05.913783 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 505/993] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:05.919521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701db815-19d7-4519-9a2a-332794af943f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:05.919803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701db815-19d7-4519-9a2a-332794af943f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:05.920075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701db815-19d7-4519-9a2a-332794af943f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:05.925530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004f156d-d4b1-4911-86dd-ffd3a053f3d3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:05.938695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-701db815-19d7-4519-9a2a-332794af943f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:05.940595 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-701db815-19d7-4519-9a2a-332794af943f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:05.967019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e090846-1555-43d1-a5de-a58073c9e02f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:05.967645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e090846-1555-43d1-a5de-a58073c9e02f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:05.967926 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e090846-1555-43d1-a5de-a58073c9e02f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:05.968200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e090846-1555-43d1-a5de-a58073c9e02f None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:05.969012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e090846-1555-43d1-a5de-a58073c9e02f None admin] RBAC: Adding query filter params (name=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:05.969653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e090846-1555-43d1-a5de-a58073c9e02f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:05.971387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e090846-1555-43d1-a5de-a58073c9e02f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:05.976998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 489/994] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:49:05 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:05.982823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e090846-1555-43d1-a5de-a58073c9e02f None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.015820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 506/995] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:05 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:06.017385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc085929-f02e-457a-92c3-3713dcb8741d None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.017700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc085929-f02e-457a-92c3-3713dcb8741d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.017947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc085929-f02e-457a-92c3-3713dcb8741d None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.022147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 507/996] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:06.024677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c9549-f289-4050-acba-906182af448d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.024958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c9549-f289-4050-acba-906182af448d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.025226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c9549-f289-4050-acba-906182af448d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.044046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d59c9549-f289-4050-acba-906182af448d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:06.045333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d59c9549-f289-4050-acba-906182af448d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:06.089805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e090846-1555-43d1-a5de-a58073c9e02f None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.118441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 508/997] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:06.133684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ebc4d-f0fd-46f8-ad54-18056fd946cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.134653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ebc4d-f0fd-46f8-ad54-18056fd946cd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.135148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ebc4d-f0fd-46f8-ad54-18056fd946cd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.135627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ebc4d-f0fd-46f8-ad54-18056fd946cd None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.136905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ebc4d-f0fd-46f8-ad54-18056fd946cd None admin] RBAC: Adding query filter params (type=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:06.137407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ebc4d-f0fd-46f8-ad54-18056fd946cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.139064 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ebc4d-f0fd-46f8-ad54-18056fd946cd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.144811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 490/998] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:49:05 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:06.150483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c9549-f289-4050-acba-906182af448d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.176890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:06.255164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c9549-f289-4050-acba-906182af448d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.289825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca2b64-6934-4432-a008-ea9bc7c220e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.290495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca2b64-6934-4432-a008-ea9bc7c220e8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.290754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca2b64-6934-4432-a008-ea9bc7c220e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.290882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ebc4d-f0fd-46f8-ad54-18056fd946cd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.291240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ca2b64-6934-4432-a008-ea9bc7c220e8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.291719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca2b64-6934-4432-a008-ea9bc7c220e8 None admin] RBAC: Adding query filter params (interface=public, service_id=5c2c7379d4b64e12a545fa900ae107bb, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:06.292094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca2b64-6934-4432-a008-ea9bc7c220e8 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.293852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ca2b64-6934-4432-a008-ea9bc7c220e8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.302413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 509/999] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Sep 13 06:49:06 2022] GET /identity/v3/endpoints?service_id=5c2c7379d4b64e12a545fa900ae107bb&interface=public®ion_id=RegionOne => generated 194 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:06.305685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca2b64-6934-4432-a008-ea9bc7c220e8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.313745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:06.396418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:06.399457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ebc4d-f0fd-46f8-ad54-18056fd946cd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.409074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ca2b64-6934-4432-a008-ea9bc7c220e8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.436199 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07687-ece1-4b8e-894b-323560b9467d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.436763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07687-ece1-4b8e-894b-323560b9467d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.437042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07687-ece1-4b8e-894b-323560b9467d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.437322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb07687-ece1-4b8e-894b-323560b9467d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.437906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb07687-ece1-4b8e-894b-323560b9467d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.439344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb07687-ece1-4b8e-894b-323560b9467d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.444512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d925f5-0dad-4f23-9c9c-3e278398a860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.445122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d925f5-0dad-4f23-9c9c-3e278398a860 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.445401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d925f5-0dad-4f23-9c9c-3e278398a860 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.445674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d925f5-0dad-4f23-9c9c-3e278398a860 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.446307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d925f5-0dad-4f23-9c9c-3e278398a860 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.447661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d925f5-0dad-4f23-9c9c-3e278398a860 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-fcb07687-ece1-4b8e-894b-323560b9467d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:06.447982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:06.449179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:06.450423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 491/1000] 10.222.0.25 () {56 vars in 1126 bytes} [Tue Sep 13 06:49:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:06.451302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07687-ece1-4b8e-894b-323560b9467d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.462145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 510/1001] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Sep 13 06:49:06 2022] GET /identity/v3/services => generated 1727 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 06:49:06.462145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d925f5-0dad-4f23-9c9c-3e278398a860 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.553543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:06.570743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d925f5-0dad-4f23-9c9c-3e278398a860 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.578109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb07687-ece1-4b8e-894b-323560b9467d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.618595 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda6558-a5b5-47e0-8508-7540f5a4f242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.619308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda6558-a5b5-47e0-8508-7540f5a4f242 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.619717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda6558-a5b5-47e0-8508-7540f5a4f242 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.620004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cda6558-a5b5-47e0-8508-7540f5a4f242 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.620779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda6558-a5b5-47e0-8508-7540f5a4f242 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:06.621099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda6558-a5b5-47e0-8508-7540f5a4f242 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.622770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cda6558-a5b5-47e0-8508-7540f5a4f242 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.625143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf754fd-0587-4eed-ba97-430b3b921811 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.625886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf754fd-0587-4eed-ba97-430b3b921811 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.626265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf754fd-0587-4eed-ba97-430b3b921811 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.626607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf754fd-0587-4eed-ba97-430b3b921811 None admin] PATH_INFO: `/v3/services/image` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.627266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf754fd-0587-4eed-ba97-430b3b921811 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.628722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bf754fd-0587-4eed-ba97-430b3b921811 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.629690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 511/1002] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:49:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:06.635657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda6558-a5b5-47e0-8508-7540f5a4f242 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2bf754fd-0587-4eed-ba97-430b3b921811 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:06.638744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:06.639775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:06.640608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 492/1003] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Sep 13 06:49:06 2022] GET /identity/v3/services/image => generated 86 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:06.645177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf754fd-0587-4eed-ba97-430b3b921811 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.659882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:06.743953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cda6558-a5b5-47e0-8508-7540f5a4f242 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.761883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf754fd-0587-4eed-ba97-430b3b921811 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.776793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0ff04-2de2-4793-b475-6c83bd758e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.777399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0ff04-2de2-4793-b475-6c83bd758e3d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.777673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0ff04-2de2-4793-b475-6c83bd758e3d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.777936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0ff04-2de2-4793-b475-6c83bd758e3d None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.778599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0ff04-2de2-4793-b475-6c83bd758e3d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:06.778903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0ff04-2de2-4793-b475-6c83bd758e3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.780270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0ff04-2de2-4793-b475-6c83bd758e3d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.785194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 512/1004] 10.222.0.25 () {56 vars in 1122 bytes} [Tue Sep 13 06:49:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:06.790579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0ff04-2de2-4793-b475-6c83bd758e3d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.797350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb88e2-be8f-4f83-818e-8b2af74ff8d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.797967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb88e2-be8f-4f83-818e-8b2af74ff8d6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.798268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb88e2-be8f-4f83-818e-8b2af74ff8d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.798535 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbb88e2-be8f-4f83-818e-8b2af74ff8d6 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.799187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb88e2-be8f-4f83-818e-8b2af74ff8d6 None admin] RBAC: Adding query filter params (name=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:06.799564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb88e2-be8f-4f83-818e-8b2af74ff8d6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.800997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbb88e2-be8f-4f83-818e-8b2af74ff8d6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.806417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 493/1005] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:06.812421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb88e2-be8f-4f83-818e-8b2af74ff8d6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:06.824860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:06.874037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:06.893050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0ff04-2de2-4793-b475-6c83bd758e3d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.923564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbb88e2-be8f-4f83-818e-8b2af74ff8d6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:06.927178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a3cd69-ffff-408e-865a-8de8ab2d10c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.927856 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3cd69-ffff-408e-865a-8de8ab2d10c3 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.928194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3cd69-ffff-408e-865a-8de8ab2d10c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.928523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a3cd69-ffff-408e-865a-8de8ab2d10c3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.929256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3cd69-ffff-408e-865a-8de8ab2d10c3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.930724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a3cd69-ffff-408e-865a-8de8ab2d10c3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.954613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 513/1006] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:49:06 2022] POST /identity/v3/endpoints => generated 325 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:49:06.967643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67881529-c3dc-44fb-8a2a-05dca11c2447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:06.968362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67881529-c3dc-44fb-8a2a-05dca11c2447 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:06.968702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67881529-c3dc-44fb-8a2a-05dca11c2447 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:06.969036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67881529-c3dc-44fb-8a2a-05dca11c2447 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:06.969741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67881529-c3dc-44fb-8a2a-05dca11c2447 None admin] RBAC: Adding query filter params (type=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:06.970157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67881529-c3dc-44fb-8a2a-05dca11c2447 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:06.971631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67881529-c3dc-44fb-8a2a-05dca11c2447 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:06.977330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 494/1007] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:06 2022] GET /identity/v3/services?type=image => generated 122 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:08.670977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205fe08-2ab6-46d4-8998-0a59823e8183 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:08.671568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205fe08-2ab6-46d4-8998-0a59823e8183 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:08.671568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205fe08-2ab6-46d4-8998-0a59823e8183 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:08.672903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 514/1008] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:08.678485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e6bf1-7e74-4f22-9ad1-cd4752a17f7a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:08.678968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e6bf1-7e74-4f22-9ad1-cd4752a17f7a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:08.679049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995e6bf1-7e74-4f22-9ad1-cd4752a17f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:08.699643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-995e6bf1-7e74-4f22-9ad1-cd4752a17f7a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:08.700849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-995e6bf1-7e74-4f22-9ad1-cd4752a17f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:08.768550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:08.775448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 495/1009] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:08 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:09.048963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205fe08-2ab6-46d4-8998-0a59823e8183 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:09.121720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236099c2-5529-4e13-84cf-2a7a46a81ca2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.122150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236099c2-5529-4e13-84cf-2a7a46a81ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.122293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236099c2-5529-4e13-84cf-2a7a46a81ca2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.123872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 496/1010] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:49:09.128853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6fb36-55fd-4a38-8c12-593bb880127a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.129137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6fb36-55fd-4a38-8c12-593bb880127a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.129411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d6fb36-55fd-4a38-8c12-593bb880127a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.147446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-63d6fb36-55fd-4a38-8c12-593bb880127a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:09.148625 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-63d6fb36-55fd-4a38-8c12-593bb880127a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:09.162275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205fe08-2ab6-46d4-8998-0a59823e8183 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:09.195452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5138f-78ae-4755-8733-d8613722c04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:09.196079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5138f-78ae-4755-8733-d8613722c04d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.196353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5138f-78ae-4755-8733-d8613722c04d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.196620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5138f-78ae-4755-8733-d8613722c04d None admin] PATH_INFO: `/v3/services/compute` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.197206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5138f-78ae-4755-8733-d8613722c04d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:09.198680 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5138f-78ae-4755-8733-d8613722c04d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-30f5138f-78ae-4755-8733-d8613722c04d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:09.207785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:09.208903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:09.210243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 515/1011] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:49:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:09.211261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0f1a-aded-4ad3-b7de-8fb0746edaa2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.211261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0f1a-aded-4ad3-b7de-8fb0746edaa2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.211261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74a0f1a-aded-4ad3-b7de-8fb0746edaa2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.212277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 516/1012] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:09.214962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0f1a-aded-4ad3-b7de-8fb0746edaa2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:09.220352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:09.223223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 497/1013] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:09 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:09.224514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a90aa-4d87-4922-b57d-abd23cd46a70 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.224798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a90aa-4d87-4922-b57d-abd23cd46a70 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.225056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446a90aa-4d87-4922-b57d-abd23cd46a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.243376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-446a90aa-4d87-4922-b57d-abd23cd46a70 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:09.245337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-446a90aa-4d87-4922-b57d-abd23cd46a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:09.317274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 498/1014] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:09 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:09.333838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74a0f1a-aded-4ad3-b7de-8fb0746edaa2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:09.368348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41e09a-179f-4148-8c18-77130b2c5525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:09.368964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41e09a-179f-4148-8c18-77130b2c5525 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.369242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41e09a-179f-4148-8c18-77130b2c5525 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.369516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f41e09a-179f-4148-8c18-77130b2c5525 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.370284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41e09a-179f-4148-8c18-77130b2c5525 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:09.370628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41e09a-179f-4148-8c18-77130b2c5525 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:09.372118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f41e09a-179f-4148-8c18-77130b2c5525 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:09.377309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 517/1015] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:49:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:09.383152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a90aa-4d87-4922-b57d-abd23cd46a70 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:09.447841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:09.492047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41e09a-179f-4148-8c18-77130b2c5525 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:09.492585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446a90aa-4d87-4922-b57d-abd23cd46a70 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:09.534320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c8d9b-7617-4847-8abe-e26a92645ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:09.534942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c8d9b-7617-4847-8abe-e26a92645ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.535217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c8d9b-7617-4847-8abe-e26a92645ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.535483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618c8d9b-7617-4847-8abe-e26a92645ff7 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.536112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c8d9b-7617-4847-8abe-e26a92645ff7 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:09.536688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c8d9b-7617-4847-8abe-e26a92645ff7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:09.537985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618c8d9b-7617-4847-8abe-e26a92645ff7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:09.548570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 499/1016] 10.222.0.25 () {56 vars in 1112 bytes} [Tue Sep 13 06:49:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:09.550466 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c8d9b-7617-4847-8abe-e26a92645ff7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:09.567900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:09.604900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f41e09a-179f-4148-8c18-77130b2c5525 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:09.639573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf78fe3-22f5-48a5-a84f-d3cf577398c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:09.640209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf78fe3-22f5-48a5-a84f-d3cf577398c9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.640488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf78fe3-22f5-48a5-a84f-d3cf577398c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.640755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf78fe3-22f5-48a5-a84f-d3cf577398c9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.641841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf78fe3-22f5-48a5-a84f-d3cf577398c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:09.643332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf78fe3-22f5-48a5-a84f-d3cf577398c9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:09.644918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 518/1017] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:09.645650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf78fe3-22f5-48a5-a84f-d3cf577398c9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:09.655326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618c8d9b-7617-4847-8abe-e26a92645ff7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:09.677754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:09.690671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160e0eb-15f1-40d6-9ba7-2e4e8fc5c02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:09.691305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160e0eb-15f1-40d6-9ba7-2e4e8fc5c02b None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.691903 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160e0eb-15f1-40d6-9ba7-2e4e8fc5c02b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.692174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c160e0eb-15f1-40d6-9ba7-2e4e8fc5c02b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.692809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160e0eb-15f1-40d6-9ba7-2e4e8fc5c02b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:09.694198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c160e0eb-15f1-40d6-9ba7-2e4e8fc5c02b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:09.718173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 500/1018] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:49:09 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:49:09.718173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160e0eb-15f1-40d6-9ba7-2e4e8fc5c02b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:09.756570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf78fe3-22f5-48a5-a84f-d3cf577398c9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:09.801523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:09.806675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9f6772-0b41-4a41-82c3-4e1a5d92b0d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:09.807296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f6772-0b41-4a41-82c3-4e1a5d92b0d5 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.807579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f6772-0b41-4a41-82c3-4e1a5d92b0d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.808201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9f6772-0b41-4a41-82c3-4e1a5d92b0d5 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.809005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f6772-0b41-4a41-82c3-4e1a5d92b0d5 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:09.810467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9f6772-0b41-4a41-82c3-4e1a5d92b0d5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:09.834224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 519/1019] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:49:09 2022] POST /identity/v3/services => generated 242 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:49:09.852254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c160e0eb-15f1-40d6-9ba7-2e4e8fc5c02b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:09.907594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8366cb8-add1-4323-b243-8f6704351795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:09.907594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366cb8-add1-4323-b243-8f6704351795 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:09.907901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366cb8-add1-4323-b243-8f6704351795 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:09.908162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8366cb8-add1-4323-b243-8f6704351795 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:09.908810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8366cb8-add1-4323-b243-8f6704351795 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:09.910453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8366cb8-add1-4323-b243-8f6704351795 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:09.946574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 501/1020] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:49:09 2022] POST /identity/v3/projects => generated 311 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:49:12.015593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343c160-9ec8-40fb-a58a-906181fde5ff None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.016142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343c160-9ec8-40fb-a58a-906181fde5ff None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.016142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9343c160-9ec8-40fb-a58a-906181fde5ff None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.016863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bc7a0-1ea2-4c51-b9f0-4df4a31589b9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.017287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bc7a0-1ea2-4c51-b9f0-4df4a31589b9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.018166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40bc7a0-1ea2-4c51-b9f0-4df4a31589b9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.018298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 520/1021] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:12.019062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 502/1022] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:49:12.023436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8318940-193b-4144-aaac-216cd797b2f1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.023737 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8318940-193b-4144-aaac-216cd797b2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.023984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8318940-193b-4144-aaac-216cd797b2f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.024463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ad16-a847-4854-8389-8192f5430533 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.024747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ad16-a847-4854-8389-8192f5430533 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.025018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0011ad16-a847-4854-8389-8192f5430533 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.044114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0011ad16-a847-4854-8389-8192f5430533 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:12.044325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b8318940-193b-4144-aaac-216cd797b2f1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:12.045340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0011ad16-a847-4854-8389-8192f5430533 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:12.045471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b8318940-193b-4144-aaac-216cd797b2f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:12.068410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:12.119929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 521/1023] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:12.122208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 503/1024] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:12.141913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3ae9b-5660-4cef-9e85-9dd54a720959 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.142238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3ae9b-5660-4cef-9e85-9dd54a720959 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.142519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a3ae9b-5660-4cef-9e85-9dd54a720959 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.143911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 522/1025] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:12.149283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ac18-ae9e-4341-a508-a38ff6d5ba7d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.149573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ac18-ae9e-4341-a508-a38ff6d5ba7d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.149848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ac18-ae9e-4341-a508-a38ff6d5ba7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.168517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b267ac18-ae9e-4341-a508-a38ff6d5ba7d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:12.169718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b267ac18-ae9e-4341-a508-a38ff6d5ba7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:12.242155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 504/1026] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:12.384890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3ae9b-5660-4cef-9e85-9dd54a720959 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:12.390472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ac18-ae9e-4341-a508-a38ff6d5ba7d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:12.480200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:12.498016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ac18-ae9e-4341-a508-a38ff6d5ba7d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:12.503606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a3ae9b-5660-4cef-9e85-9dd54a720959 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:12.526201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:12.534280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd620bd4-6a6c-4ced-a3ed-f727e033ee08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:12.534930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd620bd4-6a6c-4ced-a3ed-f727e033ee08 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.535208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd620bd4-6a6c-4ced-a3ed-f727e033ee08 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.535482 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd620bd4-6a6c-4ced-a3ed-f727e033ee08 None admin] PATH_INFO: `/v3/services/image` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.536080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd620bd4-6a6c-4ced-a3ed-f727e033ee08 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:12.537462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd620bd4-6a6c-4ced-a3ed-f727e033ee08 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:12.539239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938c326-e603-4e75-881e-9840e410970e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:12.539876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c326-e603-4e75-881e-9840e410970e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.540148 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c326-e603-4e75-881e-9840e410970e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.540414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938c326-e603-4e75-881e-9840e410970e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-bd620bd4-6a6c-4ced-a3ed-f727e033ee08 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:12.546390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:12.547419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:12.548260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 505/1027] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Sep 13 06:49:12 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:12.549063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd620bd4-6a6c-4ced-a3ed-f727e033ee08 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:12.553358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938c326-e603-4e75-881e-9840e410970e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8938c326-e603-4e75-881e-9840e410970e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:12.558578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:12.559653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 523/1028] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Sep 13 06:49:12 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:12.560495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938c326-e603-4e75-881e-9840e410970e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:12.652369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd620bd4-6a6c-4ced-a3ed-f727e033ee08 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:12.659037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938c326-e603-4e75-881e-9840e410970e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:12.685985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eaa2a-6546-4d3c-b928-d6a443a6f8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:12.686647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eaa2a-6546-4d3c-b928-d6a443a6f8c6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.686922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eaa2a-6546-4d3c-b928-d6a443a6f8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.687190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176eaa2a-6546-4d3c-b928-d6a443a6f8c6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.688278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176eaa2a-6546-4d3c-b928-d6a443a6f8c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:12.689707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176eaa2a-6546-4d3c-b928-d6a443a6f8c6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:12.691284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 506/1029] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:12.692025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eaa2a-6546-4d3c-b928-d6a443a6f8c6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:12.692500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473a4d8-76e8-4a0e-b948-f27ae0461c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:12.693117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473a4d8-76e8-4a0e-b948-f27ae0461c28 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.693395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473a4d8-76e8-4a0e-b948-f27ae0461c28 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.693669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0473a4d8-76e8-4a0e-b948-f27ae0461c28 None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.694316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473a4d8-76e8-4a0e-b948-f27ae0461c28 None admin] RBAC: Adding query filter params (name=image) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:12.694641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473a4d8-76e8-4a0e-b948-f27ae0461c28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:12.696119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0473a4d8-76e8-4a0e-b948-f27ae0461c28 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:12.705681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 524/1030] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:12 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:12.705681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473a4d8-76e8-4a0e-b948-f27ae0461c28 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:12.714225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:12.747698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:12.796925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176eaa2a-6546-4d3c-b928-d6a443a6f8c6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:12.800718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0473a4d8-76e8-4a0e-b948-f27ae0461c28 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:12.831178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64583662-7518-405c-a04f-b8643ab8b926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:12.831813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64583662-7518-405c-a04f-b8643ab8b926 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.832090 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64583662-7518-405c-a04f-b8643ab8b926 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.832407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64583662-7518-405c-a04f-b8643ab8b926 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.833010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64583662-7518-405c-a04f-b8643ab8b926 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:12.833309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64583662-7518-405c-a04f-b8643ab8b926 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:12.834714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64583662-7518-405c-a04f-b8643ab8b926 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:12.835451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0a16b-ec27-45e9-b6d7-90fc2f70a6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:12.836065 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0a16b-ec27-45e9-b6d7-90fc2f70a6aa None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:12.836347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0a16b-ec27-45e9-b6d7-90fc2f70a6aa None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:12.836885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d0a16b-ec27-45e9-b6d7-90fc2f70a6aa None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:12.837269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0a16b-ec27-45e9-b6d7-90fc2f70a6aa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:12.838741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d0a16b-ec27-45e9-b6d7-90fc2f70a6aa None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:12.842718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 507/1031] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Sep 13 06:49:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:12.843503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64583662-7518-405c-a04f-b8643ab8b926 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:12.862760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 525/1032] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:49:12 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:49:12.863588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0a16b-ec27-45e9-b6d7-90fc2f70a6aa None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:12.939382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:12.966070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64583662-7518-405c-a04f-b8643ab8b926 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:12.978400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d0a16b-ec27-45e9-b6d7-90fc2f70a6aa None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:12.980389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:13.007253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50b2ff-9c83-49e0-beef-e1953b9a2da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:13.007894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50b2ff-9c83-49e0-beef-e1953b9a2da7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:13.008173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50b2ff-9c83-49e0-beef-e1953b9a2da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:13.008482 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50b2ff-9c83-49e0-beef-e1953b9a2da7 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:13.009129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50b2ff-9c83-49e0-beef-e1953b9a2da7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:13.009441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50b2ff-9c83-49e0-beef-e1953b9a2da7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:13.011045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50b2ff-9c83-49e0-beef-e1953b9a2da7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:13.017001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 508/1033] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:12 2022] GET /identity/v3/services?type=image => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:13.023548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50b2ff-9c83-49e0-beef-e1953b9a2da7 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:13.025641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1c682-6065-48bf-bdc5-6b64249c1ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:13.026429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1c682-6065-48bf-bdc5-6b64249c1ca3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:13.026714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1c682-6065-48bf-bdc5-6b64249c1ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:13.026975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d1c682-6065-48bf-bdc5-6b64249c1ca3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:13.038603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1c682-6065-48bf-bdc5-6b64249c1ca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:13.040083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d1c682-6065-48bf-bdc5-6b64249c1ca3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-f9d1c682-6065-48bf-bdc5-6b64249c1ca3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:13.055546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:13.056854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:49:13.059159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:13.059159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:13.059159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:13.059159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 526/1034] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:13.064747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1c682-6065-48bf-bdc5-6b64249c1ca3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:13.143157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50b2ff-9c83-49e0-beef-e1953b9a2da7 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:13.180920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d1c682-6065-48bf-bdc5-6b64249c1ca3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:13.181372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0b951-c93b-4f99-9820-26b5f456d07c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:13.181993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0b951-c93b-4f99-9820-26b5f456d07c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:13.182402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0b951-c93b-4f99-9820-26b5f456d07c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:13.182682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c0b951-c93b-4f99-9820-26b5f456d07c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:13.183746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0b951-c93b-4f99-9820-26b5f456d07c None admin] RBAC: Adding query filter params (interface=public, service_id=2e35461e75c6438c8849a8155b72de2a, region_id=RegionOne) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:13.184045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0b951-c93b-4f99-9820-26b5f456d07c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:13.185424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c0b951-c93b-4f99-9820-26b5f456d07c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:13.190658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 509/1035] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Sep 13 06:49:13 2022] GET /identity/v3/endpoints?service_id=2e35461e75c6438c8849a8155b72de2a&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:13.196361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0b951-c93b-4f99-9820-26b5f456d07c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:13.208405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:13.223304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ef6ec-05fb-4ebe-91d2-9a974b1b36a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:13.224027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ef6ec-05fb-4ebe-91d2-9a974b1b36a8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:13.224300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ef6ec-05fb-4ebe-91d2-9a974b1b36a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:13.224573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23ef6ec-05fb-4ebe-91d2-9a974b1b36a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:13.225217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ef6ec-05fb-4ebe-91d2-9a974b1b36a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:13.225530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ef6ec-05fb-4ebe-91d2-9a974b1b36a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:13.226938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23ef6ec-05fb-4ebe-91d2-9a974b1b36a8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:13.240306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 527/1036] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:49:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:13.245819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ef6ec-05fb-4ebe-91d2-9a974b1b36a8 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:13.263503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:13.308823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c0b951-c93b-4f99-9820-26b5f456d07c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:13.346488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248677c4-bd97-49ca-8875-2321e881b239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:13.347113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248677c4-bd97-49ca-8875-2321e881b239 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:13.347396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248677c4-bd97-49ca-8875-2321e881b239 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:13.347675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-248677c4-bd97-49ca-8875-2321e881b239 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:13.348285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248677c4-bd97-49ca-8875-2321e881b239 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:13.349667 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-248677c4-bd97-49ca-8875-2321e881b239 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:13.353730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23ef6ec-05fb-4ebe-91d2-9a974b1b36a8 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:13.356991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 510/1037] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Sep 13 06:49:13 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 06:49:13.395224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90986fe5-77b2-4ee0-8856-f65c95dcf5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:13.395908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90986fe5-77b2-4ee0-8856-f65c95dcf5da None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:13.396184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90986fe5-77b2-4ee0-8856-f65c95dcf5da None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:13.396419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90986fe5-77b2-4ee0-8856-f65c95dcf5da None admin] PATH_INFO: `/v3/users/nova` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:13.405319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90986fe5-77b2-4ee0-8856-f65c95dcf5da None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:13.406876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90986fe5-77b2-4ee0-8856-f65c95dcf5da None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-90986fe5-77b2-4ee0-8856-f65c95dcf5da None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:49:13.420249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:13.421308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:13.422310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 528/1038] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Sep 13 06:49:13 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:13.427776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248677c4-bd97-49ca-8875-2321e881b239 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:13.511998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:13.546961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-248677c4-bd97-49ca-8875-2321e881b239 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:13.581778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f0cc1-1f19-4348-a9d5-eebd6ea7874b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:13.582427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f0cc1-1f19-4348-a9d5-eebd6ea7874b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:13.582703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f0cc1-1f19-4348-a9d5-eebd6ea7874b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:13.582970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f0cc1-1f19-4348-a9d5-eebd6ea7874b None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:13.583697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f0cc1-1f19-4348-a9d5-eebd6ea7874b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:13.584002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f0cc1-1f19-4348-a9d5-eebd6ea7874b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:13.585439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f0cc1-1f19-4348-a9d5-eebd6ea7874b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:13.614281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 511/1039] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Sep 13 06:49:13 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:13.615904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90986fe5-77b2-4ee0-8856-f65c95dcf5da None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:13.694410 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:13.721137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90986fe5-77b2-4ee0-8856-f65c95dcf5da None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:13.758387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0c922-63c4-4449-b334-970ec09ff8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:13.759010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0c922-63c4-4449-b334-970ec09ff8d9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:13.759290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0c922-63c4-4449-b334-970ec09ff8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:13.759562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c0c922-63c4-4449-b334-970ec09ff8d9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:13.830065 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0c922-63c4-4449-b334-970ec09ff8d9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:13.831486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c0c922-63c4-4449-b334-970ec09ff8d9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:13.905341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:13.932375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 529/1040] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:13 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 06:49:13.938685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f0cc1-1f19-4348-a9d5-eebd6ea7874b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:14.038294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f0cc1-1f19-4348-a9d5-eebd6ea7874b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:14.067727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:14.075446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b93da8-5218-4e6f-bdfe-62670782a53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:14.076098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b93da8-5218-4e6f-bdfe-62670782a53a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:14.076378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b93da8-5218-4e6f-bdfe-62670782a53a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:14.076651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b93da8-5218-4e6f-bdfe-62670782a53a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:14.087457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b93da8-5218-4e6f-bdfe-62670782a53a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:14.088932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b93da8-5218-4e6f-bdfe-62670782a53a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-41b93da8-5218-4e6f-bdfe-62670782a53a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:14.103898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:14.104938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:49:14.106018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:14.106018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:14.106018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:14.109848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 512/1041] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:14.111320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0c922-63c4-4449-b334-970ec09ff8d9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:14.222446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c0c922-63c4-4449-b334-970ec09ff8d9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:14.259344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3843d-c960-4716-aa88-0b4f7dfe6b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:14.259987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3843d-c960-4716-aa88-0b4f7dfe6b28 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:14.260258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3843d-c960-4716-aa88-0b4f7dfe6b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:14.260525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d3843d-c960-4716-aa88-0b4f7dfe6b28 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:14.261258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3843d-c960-4716-aa88-0b4f7dfe6b28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:14.261559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3843d-c960-4716-aa88-0b4f7dfe6b28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:14.262978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d3843d-c960-4716-aa88-0b4f7dfe6b28 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:14.278929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 530/1042] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:49:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:14.280637 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b93da8-5218-4e6f-bdfe-62670782a53a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:14.383762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b93da8-5218-4e6f-bdfe-62670782a53a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:14.417065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0a14b-d199-46f6-a055-8f4da72294b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:14.417704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0a14b-d199-46f6-a055-8f4da72294b4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:14.418012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0a14b-d199-46f6-a055-8f4da72294b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:14.418284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba0a14b-d199-46f6-a055-8f4da72294b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:14.425054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0a14b-d199-46f6-a055-8f4da72294b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:14.426538 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba0a14b-d199-46f6-a055-8f4da72294b4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-eba0a14b-d199-46f6-a055-8f4da72294b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:49:14.438439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:14.439313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:14.440205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:49:14.440205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:49:14.440205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:49:14.440205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:49:14.440205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:49:14.440205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:49:14.440205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:49:14.440205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:14.442394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 513/1043] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:49:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:14.445526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3843d-c960-4716-aa88-0b4f7dfe6b28 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:14.482145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:14.553181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d3843d-c960-4716-aa88-0b4f7dfe6b28 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:14.588574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd230375-f8ea-427b-bd26-29c8db181c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:14.589277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230375-f8ea-427b-bd26-29c8db181c79 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:14.589556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230375-f8ea-427b-bd26-29c8db181c79 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:14.589828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd230375-f8ea-427b-bd26-29c8db181c79 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:14.590557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd230375-f8ea-427b-bd26-29c8db181c79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:14.590866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd230375-f8ea-427b-bd26-29c8db181c79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:14.592346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd230375-f8ea-427b-bd26-29c8db181c79 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:14.612312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 531/1044] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:49:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:14.613867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0a14b-d199-46f6-a055-8f4da72294b4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:14.660655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:14.720224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba0a14b-d199-46f6-a055-8f4da72294b4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:14.754651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc89461-cd25-4a45-b406-fd1276ba0f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:14.755714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89461-cd25-4a45-b406-fd1276ba0f92 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:14.756190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89461-cd25-4a45-b406-fd1276ba0f92 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:14.756651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc89461-cd25-4a45-b406-fd1276ba0f92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:14.757556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc89461-cd25-4a45-b406-fd1276ba0f92 None admin] RBAC: Adding query filter params (role.id=d0cb30355e064a3d8562073f93479915, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=87f3a9ffb88f4faa85edc3dabce0c8f5) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:14.758054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc89461-cd25-4a45-b406-fd1276ba0f92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:14.759721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc89461-cd25-4a45-b406-fd1276ba0f92 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:14.765731 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 514/1045] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:49:14 2022] GET /identity/v3/role_assignments?user.id=87f3a9ffb88f4faa85edc3dabce0c8f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=d0cb30355e064a3d8562073f93479915 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:15.082781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24600fe5-4533-4c1f-bc35-2db1f9b415c4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:15.082968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24600fe5-4533-4c1f-bc35-2db1f9b415c4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:15.083240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24600fe5-4533-4c1f-bc35-2db1f9b415c4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:15.084616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 532/1046] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:15.090141 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9786ada-5abd-4921-9daf-cda082fa42ab None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:15.090621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9786ada-5abd-4921-9daf-cda082fa42ab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:15.090706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9786ada-5abd-4921-9daf-cda082fa42ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:15.109051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a9786ada-5abd-4921-9daf-cda082fa42ab None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:15.110286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a9786ada-5abd-4921-9daf-cda082fa42ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:15.186292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 515/1047] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:15.450000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24600fe5-4533-4c1f-bc35-2db1f9b415c4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:15.517252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c95dba-b0ce-4459-9b33-3f5e508b80cf None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:15.517495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c95dba-b0ce-4459-9b33-3f5e508b80cf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:15.517778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c95dba-b0ce-4459-9b33-3f5e508b80cf None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:15.519177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 516/1048] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:49:15.524381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e9022-5cb8-484f-8304-13db53780b57 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:15.524660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e9022-5cb8-484f-8304-13db53780b57 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:15.524959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e9022-5cb8-484f-8304-13db53780b57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:15.543394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-122e9022-5cb8-484f-8304-13db53780b57 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:15.544592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-122e9022-5cb8-484f-8304-13db53780b57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:15.551175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:15.562693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24600fe5-4533-4c1f-bc35-2db1f9b415c4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:15.578543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:15.603695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a2198-1978-4545-84cd-9390d651fe10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:15.604340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a2198-1978-4545-84cd-9390d651fe10 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:15.604695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a2198-1978-4545-84cd-9390d651fe10 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:15.604975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836a2198-1978-4545-84cd-9390d651fe10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:15.620133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 517/1049] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:15 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:15.622222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836a2198-1978-4545-84cd-9390d651fe10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-836a2198-1978-4545-84cd-9390d651fe10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:15.626376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:15.627347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:15.628223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:15.629609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 533/1050] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:49:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:15.633893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e9022-5cb8-484f-8304-13db53780b57 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:15.736190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e9022-5cb8-484f-8304-13db53780b57 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:15.769491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9df4a-7927-47bf-828c-8ebdd062a04c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:15.770128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9df4a-7927-47bf-828c-8ebdd062a04c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:15.770423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9df4a-7927-47bf-828c-8ebdd062a04c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:15.770680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb9df4a-7927-47bf-828c-8ebdd062a04c None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:15.771344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9df4a-7927-47bf-828c-8ebdd062a04c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:15.771644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9df4a-7927-47bf-828c-8ebdd062a04c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:15.773143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb9df4a-7927-47bf-828c-8ebdd062a04c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:15.786770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 518/1051] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:49:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:15.788268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a2198-1978-4545-84cd-9390d651fe10 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:15.884866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9df4a-7927-47bf-828c-8ebdd062a04c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:15.894575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836a2198-1978-4545-84cd-9390d651fe10 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:15.920138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:15.931666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e882-4719-4b12-a57f-0bc95d01da71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:15.932308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e882-4719-4b12-a57f-0bc95d01da71 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:15.932587 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e882-4719-4b12-a57f-0bc95d01da71 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:15.933253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3454e882-4719-4b12-a57f-0bc95d01da71 None admin] PATH_INFO: `/v3/users/5dee8b64cc58448cab1a78b1d53ca562` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:15.947334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e882-4719-4b12-a57f-0bc95d01da71 None admin] RBAC: Authorizing `identity:get_user(user_id=5dee8b64cc58448cab1a78b1d53ca562)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:15.948950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3454e882-4719-4b12-a57f-0bc95d01da71 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:15.950774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 534/1052] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:15 2022] GET /identity/v3/users/5dee8b64cc58448cab1a78b1d53ca562 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:15.956225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e882-4719-4b12-a57f-0bc95d01da71 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:15.976306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:16.002683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb9df4a-7927-47bf-828c-8ebdd062a04c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:16.037904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014528a-28ea-44bb-a55e-719812284377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:16.038508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014528a-28ea-44bb-a55e-719812284377 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:16.038785 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014528a-28ea-44bb-a55e-719812284377 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:16.039047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6014528a-28ea-44bb-a55e-719812284377 None admin] PATH_INFO: `/v3/services/image` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:16.039622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014528a-28ea-44bb-a55e-719812284377 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:16.041001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6014528a-28ea-44bb-a55e-719812284377 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6014528a-28ea-44bb-a55e-719812284377 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:16.050046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:16.051142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:16.051991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 519/1053] 10.222.0.25 () {56 vars in 1105 bytes} [Tue Sep 13 06:49:15 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:16.059690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014528a-28ea-44bb-a55e-719812284377 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:16.070640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3454e882-4719-4b12-a57f-0bc95d01da71 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:16.104318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c961cf-6e50-470a-958b-4e6f2521eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:16.105017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c961cf-6e50-470a-958b-4e6f2521eff6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:16.105300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c961cf-6e50-470a-958b-4e6f2521eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:16.105577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94c961cf-6e50-470a-958b-4e6f2521eff6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:16.164824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:16.176533 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6014528a-28ea-44bb-a55e-719812284377 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:16.176756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c961cf-6e50-470a-958b-4e6f2521eff6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:16.178245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94c961cf-6e50-470a-958b-4e6f2521eff6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:16.192480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:16.218304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb45e6b-2d94-45ef-8eba-a2203297f776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:16.219041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45e6b-2d94-45ef-8eba-a2203297f776 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:16.219321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45e6b-2d94-45ef-8eba-a2203297f776 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:16.219593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45e6b-2d94-45ef-8eba-a2203297f776 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:16.220220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45e6b-2d94-45ef-8eba-a2203297f776 None admin] RBAC: Adding query filter params (name=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:16.220635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45e6b-2d94-45ef-8eba-a2203297f776 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:16.222008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45e6b-2d94-45ef-8eba-a2203297f776 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:16.233508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 520/1054] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:16 2022] GET /identity/v3/services?name=image => generated 122 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:16.234598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb45e6b-2d94-45ef-8eba-a2203297f776 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:16.282865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 535/1055] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:15 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:49:16.288328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c961cf-6e50-470a-958b-4e6f2521eff6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:16.338441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb45e6b-2d94-45ef-8eba-a2203297f776 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:16.373934 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228b55c-5bd1-422b-9b6e-24978c7675cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:16.374569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228b55c-5bd1-422b-9b6e-24978c7675cd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:16.374871 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228b55c-5bd1-422b-9b6e-24978c7675cd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:16.375152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d228b55c-5bd1-422b-9b6e-24978c7675cd None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:16.375749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228b55c-5bd1-422b-9b6e-24978c7675cd None admin] RBAC: Adding query filter params (type=image) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:16.376083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228b55c-5bd1-422b-9b6e-24978c7675cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:16.377447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d228b55c-5bd1-422b-9b6e-24978c7675cd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:16.382248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 521/1056] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:16 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:16.387963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228b55c-5bd1-422b-9b6e-24978c7675cd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:16.391414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94c961cf-6e50-470a-958b-4e6f2521eff6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:16.405214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:16.416436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:16.432612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cadd38-d173-47d5-9153-f5480d5b1146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:16.433618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cadd38-d173-47d5-9153-f5480d5b1146 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:16.433897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cadd38-d173-47d5-9153-f5480d5b1146 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:16.434346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cadd38-d173-47d5-9153-f5480d5b1146 None admin] PATH_INFO: `/v3/projects/5f4904a204af495ea615f536dcc5268b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:16.435516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cadd38-d173-47d5-9153-f5480d5b1146 None admin] RBAC: Authorizing `identity:get_project(project_id=5f4904a204af495ea615f536dcc5268b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:16.436978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cadd38-d173-47d5-9153-f5480d5b1146 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:16.438692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 536/1057] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:49:16 2022] GET /identity/v3/projects/5f4904a204af495ea615f536dcc5268b => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:16.444780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cadd38-d173-47d5-9153-f5480d5b1146 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:16.509441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d228b55c-5bd1-422b-9b6e-24978c7675cd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:16.546559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d707bdd-f662-4d6f-ab36-aee313dc7c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:16.547255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d707bdd-f662-4d6f-ab36-aee313dc7c38 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:16.547603 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d707bdd-f662-4d6f-ab36-aee313dc7c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:16.547945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d707bdd-f662-4d6f-ab36-aee313dc7c38 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:16.548181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cadd38-d173-47d5-9153-f5480d5b1146 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:16.548806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d707bdd-f662-4d6f-ab36-aee313dc7c38 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:16.550281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d707bdd-f662-4d6f-ab36-aee313dc7c38 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:16.578129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 522/1058] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:49:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:49:16.592179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e91f9c-0cbb-41bc-9d56-048f81982db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:16.592810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e91f9c-0cbb-41bc-9d56-048f81982db8 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:16.593086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e91f9c-0cbb-41bc-9d56-048f81982db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:16.593344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e91f9c-0cbb-41bc-9d56-048f81982db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:16.594106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e91f9c-0cbb-41bc-9d56-048f81982db8 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=5f4904a204af495ea615f536dcc5268b, user.id=5dee8b64cc58448cab1a78b1d53ca562) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:16.594407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e91f9c-0cbb-41bc-9d56-048f81982db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:16.595864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e91f9c-0cbb-41bc-9d56-048f81982db8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:16.602592 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 537/1059] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:49:16 2022] GET /identity/v3/role_assignments?user.id=5dee8b64cc58448cab1a78b1d53ca562&scope.project.id=5f4904a204af495ea615f536dcc5268b&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:16.924034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd787223-b5c3-4c0f-8a41-9ed089b16abb None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:16.924289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd787223-b5c3-4c0f-8a41-9ed089b16abb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:16.924552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd787223-b5c3-4c0f-8a41-9ed089b16abb None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:16.926160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 523/1060] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:16.931398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b6132-2dff-42b3-aff6-13e2540cb8e3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:16.931655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b6132-2dff-42b3-aff6-13e2540cb8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:16.931963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd1b6132-2dff-42b3-aff6-13e2540cb8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:16.951048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cd1b6132-2dff-42b3-aff6-13e2540cb8e3 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:16.952263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cd1b6132-2dff-42b3-aff6-13e2540cb8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:16.987865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:17.025052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 538/1061] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:17.294059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd787223-b5c3-4c0f-8a41-9ed089b16abb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:17.377570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:17.403444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd787223-b5c3-4c0f-8a41-9ed089b16abb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:17.437693 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c598d3-aa39-4c93-9d90-79619833b989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:17.438356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c598d3-aa39-4c93-9d90-79619833b989 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:17.438639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c598d3-aa39-4c93-9d90-79619833b989 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:17.438912 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c598d3-aa39-4c93-9d90-79619833b989 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:17.449649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c598d3-aa39-4c93-9d90-79619833b989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-19c598d3-aa39-4c93-9d90-79619833b989 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:17.454534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:17.455567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:49:17.456494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:17.458423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 524/1062] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Sep 13 06:49:17 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:17.461846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b6132-2dff-42b3-aff6-13e2540cb8e3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:17.568260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd1b6132-2dff-42b3-aff6-13e2540cb8e3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:17.612394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf02744-dd20-4515-ac87-c3ba8928bb14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:17.613012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf02744-dd20-4515-ac87-c3ba8928bb14 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:17.613290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf02744-dd20-4515-ac87-c3ba8928bb14 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:17.613558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf02744-dd20-4515-ac87-c3ba8928bb14 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:17.614263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf02744-dd20-4515-ac87-c3ba8928bb14 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:17.614568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf02744-dd20-4515-ac87-c3ba8928bb14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:17.615943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf02744-dd20-4515-ac87-c3ba8928bb14 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:17.624540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 539/1063] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Sep 13 06:49:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:17.630232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c598d3-aa39-4c93-9d90-79619833b989 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:17.729069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c598d3-aa39-4c93-9d90-79619833b989 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:17.762667 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747c7c2-05bc-443a-90bb-d6d5b8877926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:17.763306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747c7c2-05bc-443a-90bb-d6d5b8877926 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:17.763584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747c7c2-05bc-443a-90bb-d6d5b8877926 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:17.763856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f747c7c2-05bc-443a-90bb-d6d5b8877926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:17.766599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:17.775786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747c7c2-05bc-443a-90bb-d6d5b8877926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:17.777476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f747c7c2-05bc-443a-90bb-d6d5b8877926 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f747c7c2-05bc-443a-90bb-d6d5b8877926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:17.791727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:17.792917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:49:17.793820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:17.793820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:17.793820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:17.798461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 525/1064] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:17.799801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf02744-dd20-4515-ac87-c3ba8928bb14 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:17.833861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:17.904231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf02744-dd20-4515-ac87-c3ba8928bb14 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:17.937687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21445bc-3d29-43b7-8622-ed49e43a5d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:17.938316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21445bc-3d29-43b7-8622-ed49e43a5d6b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:17.938588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21445bc-3d29-43b7-8622-ed49e43a5d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:17.938855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21445bc-3d29-43b7-8622-ed49e43a5d6b None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:17.939476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21445bc-3d29-43b7-8622-ed49e43a5d6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:17.939776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21445bc-3d29-43b7-8622-ed49e43a5d6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:17.941148 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21445bc-3d29-43b7-8622-ed49e43a5d6b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:17.953288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 540/1065] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:49:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:17.958845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747c7c2-05bc-443a-90bb-d6d5b8877926 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:18.061812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f747c7c2-05bc-443a-90bb-d6d5b8877926 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:18.094718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43964b-8f8f-4f9e-a8c6-afff60ef58f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:18.095344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43964b-8f8f-4f9e-a8c6-afff60ef58f1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:18.095627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43964b-8f8f-4f9e-a8c6-afff60ef58f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:18.095900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d43964b-8f8f-4f9e-a8c6-afff60ef58f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:18.103449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43964b-8f8f-4f9e-a8c6-afff60ef58f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:18.104997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d43964b-8f8f-4f9e-a8c6-afff60ef58f1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-9d43964b-8f8f-4f9e-a8c6-afff60ef58f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:49:18.116132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:18.117094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:18.118125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 526/1066] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Sep 13 06:49:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:18.124700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21445bc-3d29-43b7-8622-ed49e43a5d6b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:18.250805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:18.259733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21445bc-3d29-43b7-8622-ed49e43a5d6b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:18.296640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe6cae-f93b-4f7a-98a5-b9a19a68089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:18.302098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe6cae-f93b-4f7a-98a5-b9a19a68089e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:18.302098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe6cae-f93b-4f7a-98a5-b9a19a68089e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:18.302098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbe6cae-f93b-4f7a-98a5-b9a19a68089e None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:18.302098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe6cae-f93b-4f7a-98a5-b9a19a68089e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:18.302098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe6cae-f93b-4f7a-98a5-b9a19a68089e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:18.302098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbe6cae-f93b-4f7a-98a5-b9a19a68089e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:18.319335 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 541/1067] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Sep 13 06:49:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:18.327196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43964b-8f8f-4f9e-a8c6-afff60ef58f1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:18.378769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:18.433951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d43964b-8f8f-4f9e-a8c6-afff60ef58f1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:18.467781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c311f-3640-4393-a977-c070ac507e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:18.468577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c311f-3640-4393-a977-c070ac507e81 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:18.468948 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c311f-3640-4393-a977-c070ac507e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:18.469313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7c311f-3640-4393-a977-c070ac507e81 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:18.480760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c311f-3640-4393-a977-c070ac507e81 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:18.482466 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7c311f-3640-4393-a977-c070ac507e81 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1f7c311f-3640-4393-a977-c070ac507e81 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:18.499457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:18.500655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:49:18.501651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:18.501651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:18.501651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:18.508090 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 527/1068] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:18.508240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe6cae-f93b-4f7a-98a5-b9a19a68089e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:18.636206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbe6cae-f93b-4f7a-98a5-b9a19a68089e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:18.675477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee19f2b-e18e-474a-acaa-6b3dd860d411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:18.676110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19f2b-e18e-474a-acaa-6b3dd860d411 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:18.676394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19f2b-e18e-474a-acaa-6b3dd860d411 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:18.676665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee19f2b-e18e-474a-acaa-6b3dd860d411 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:18.677312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee19f2b-e18e-474a-acaa-6b3dd860d411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:18.677626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee19f2b-e18e-474a-acaa-6b3dd860d411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:18.679085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee19f2b-e18e-474a-acaa-6b3dd860d411 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:18.693443 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 542/1069] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:49:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:18.699430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c311f-3640-4393-a977-c070ac507e81 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:18.753796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f237d-9f1b-4ae2-9821-dd36b75f8637 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:18.754185 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f237d-9f1b-4ae2-9821-dd36b75f8637 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:18.754382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f237d-9f1b-4ae2-9821-dd36b75f8637 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:18.755881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 543/1070] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:18.760722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982ad19-215e-4d62-a918-0f41e8c7210e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:18.761004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982ad19-215e-4d62-a918-0f41e8c7210e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:18.761271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1982ad19-215e-4d62-a918-0f41e8c7210e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:18.779942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1982ad19-215e-4d62-a918-0f41e8c7210e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:18.781156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1982ad19-215e-4d62-a918-0f41e8c7210e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:18.784459 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:18.803348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:18.806838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7c311f-3640-4393-a977-c070ac507e81 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:18.840314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b9015-0b2c-4d29-9812-9e247f297e04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:18.840945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b9015-0b2c-4d29-9812-9e247f297e04 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:18.841221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b9015-0b2c-4d29-9812-9e247f297e04 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:18.841488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b9015-0b2c-4d29-9812-9e247f297e04 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:18.847943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b9015-0b2c-4d29-9812-9e247f297e04 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:18.849375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867b9015-0b2c-4d29-9812-9e247f297e04 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:18.851177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 544/1071] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:18.852450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c62fd6-9711-4270-9133-56a8c4f1f7b8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:18.852741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c62fd6-9711-4270-9133-56a8c4f1f7b8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:18.853016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c62fd6-9711-4270-9133-56a8c4f1f7b8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:18.854564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 545/1072] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:18.859962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c1b72-6719-4507-81c8-d82875faa455 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:18.860249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c1b72-6719-4507-81c8-d82875faa455 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:18.860529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50c1b72-6719-4507-81c8-d82875faa455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-867b9015-0b2c-4d29-9812-9e247f297e04 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:49:18.863988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:18.864904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:18.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:49:18.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:49:18.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:49:18.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:49:18.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:49:18.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:49:18.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:49:18.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:18.867061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 528/1073] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:49:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:18.870942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b9015-0b2c-4d29-9812-9e247f297e04 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:18.892393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d50c1b72-6719-4507-81c8-d82875faa455 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:18.893500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d50c1b72-6719-4507-81c8-d82875faa455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:18.964568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 546/1074] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:18.985580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b9015-0b2c-4d29-9812-9e247f297e04 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:19.020305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb453b0-4689-4eb4-8f63-42fe7c4c4deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:19.020913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb453b0-4689-4eb4-8f63-42fe7c4c4deb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:19.021186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb453b0-4689-4eb4-8f63-42fe7c4c4deb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:19.021451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb453b0-4689-4eb4-8f63-42fe7c4c4deb None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:19.022162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb453b0-4689-4eb4-8f63-42fe7c4c4deb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:19.022465 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb453b0-4689-4eb4-8f63-42fe7c4c4deb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:19.023884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb453b0-4689-4eb4-8f63-42fe7c4c4deb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:19.036407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 529/1075] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:49:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:19.042425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c1b72-6719-4507-81c8-d82875faa455 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:19.111712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:19.118168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb453b0-4689-4eb4-8f63-42fe7c4c4deb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:19.145190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50c1b72-6719-4507-81c8-d82875faa455 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:19.148741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:19.181982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b62c7-33ca-4574-8738-643d78ae1d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:19.183190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b62c7-33ca-4574-8738-643d78ae1d2f None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:19.183470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b62c7-33ca-4574-8738-643d78ae1d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:19.183752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1b62c7-33ca-4574-8738-643d78ae1d2f None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/87f3a9ffb88f4faa85edc3dabce0c8f5/roles/d0cb30355e064a3d8562073f93479915` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:19.185981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b62c7-33ca-4574-8738-643d78ae1d2f None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=87f3a9ffb88f4faa85edc3dabce0c8f5, role_id=d0cb30355e064a3d8562073f93479915)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:19.187477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1b62c7-33ca-4574-8738-643d78ae1d2f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:19.202897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 547/1076] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:49:19 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/87f3a9ffb88f4faa85edc3dabce0c8f5/roles/d0cb30355e064a3d8562073f93479915 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:49:19.227835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b62c7-33ca-4574-8738-643d78ae1d2f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:19.259423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb453b0-4689-4eb4-8f63-42fe7c4c4deb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:19.301523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f56c2-d815-4670-8316-e2eabd389453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:19.302190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f56c2-d815-4670-8316-e2eabd389453 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:19.302474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f56c2-d815-4670-8316-e2eabd389453 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:19.302750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f56c2-d815-4670-8316-e2eabd389453 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:19.345785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1b62c7-33ca-4574-8738-643d78ae1d2f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:19.379201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:19.380758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f56c2-d815-4670-8316-e2eabd389453 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:19.382755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f56c2-d815-4670-8316-e2eabd389453 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:19.395167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905566b5-ccc8-4223-a5d4-d190780cf106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:19.395881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905566b5-ccc8-4223-a5d4-d190780cf106 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:19.396104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905566b5-ccc8-4223-a5d4-d190780cf106 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:19.396388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905566b5-ccc8-4223-a5d4-d190780cf106 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:19.408271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:19.410217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905566b5-ccc8-4223-a5d4-d190780cf106 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-905566b5-ccc8-4223-a5d4-d190780cf106 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:19.415310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:19.416294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:19.417184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 548/1077] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:49:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:19.422936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905566b5-ccc8-4223-a5d4-d190780cf106 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:19.493828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 530/1078] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:49:19.499558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f56c2-d815-4670-8316-e2eabd389453 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:19.528806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905566b5-ccc8-4223-a5d4-d190780cf106 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:19.567448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44670c37-ae83-4a00-b3d4-09f8638cbc61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:19.568342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44670c37-ae83-4a00-b3d4-09f8638cbc61 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:19.568342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44670c37-ae83-4a00-b3d4-09f8638cbc61 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:19.568612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44670c37-ae83-4a00-b3d4-09f8638cbc61 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:19.569250 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44670c37-ae83-4a00-b3d4-09f8638cbc61 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:19.569553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44670c37-ae83-4a00-b3d4-09f8638cbc61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:19.570994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44670c37-ae83-4a00-b3d4-09f8638cbc61 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:19.580594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 549/1079] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:49:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:19.586021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44670c37-ae83-4a00-b3d4-09f8638cbc61 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:19.604206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f56c2-d815-4670-8316-e2eabd389453 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:19.629828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:19.644420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:19.646626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54b868-a7e6-4c98-843e-3ded3bf2b1bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:19.647267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54b868-a7e6-4c98-843e-3ded3bf2b1bf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:19.647547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54b868-a7e6-4c98-843e-3ded3bf2b1bf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:19.647818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e54b868-a7e6-4c98-843e-3ded3bf2b1bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:19.658980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54b868-a7e6-4c98-843e-3ded3bf2b1bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:19.660431 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e54b868-a7e6-4c98-843e-3ded3bf2b1bf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6e54b868-a7e6-4c98-843e-3ded3bf2b1bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:19.678708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:19.679808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:49:19.680703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:19.680703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:19.680703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:19.680703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 531/1080] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:19.686531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54b868-a7e6-4c98-843e-3ded3bf2b1bf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:19.695265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44670c37-ae83-4a00-b3d4-09f8638cbc61 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:19.732163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c800a74-f433-478b-87f5-dc8c19346fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:19.733316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c800a74-f433-478b-87f5-dc8c19346fab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:19.733599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c800a74-f433-478b-87f5-dc8c19346fab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:19.733873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c800a74-f433-478b-87f5-dc8c19346fab None admin] PATH_INFO: `/v3/users/5dee8b64cc58448cab1a78b1d53ca562` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:19.735391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c800a74-f433-478b-87f5-dc8c19346fab None admin] RBAC: Authorizing `identity:get_user(user_id=5dee8b64cc58448cab1a78b1d53ca562)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:19.737164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c800a74-f433-478b-87f5-dc8c19346fab None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:19.743639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 550/1081] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:19 2022] GET /identity/v3/users/5dee8b64cc58448cab1a78b1d53ca562 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:19.744579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c800a74-f433-478b-87f5-dc8c19346fab None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:19.802840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e54b868-a7e6-4c98-843e-3ded3bf2b1bf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:19.843844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524892e-a5ef-4a07-a14c-caf84f9a4abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:19.844464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524892e-a5ef-4a07-a14c-caf84f9a4abd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:19.844742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524892e-a5ef-4a07-a14c-caf84f9a4abd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:19.845011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524892e-a5ef-4a07-a14c-caf84f9a4abd None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:19.845646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524892e-a5ef-4a07-a14c-caf84f9a4abd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:19.845942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524892e-a5ef-4a07-a14c-caf84f9a4abd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:19.847347 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524892e-a5ef-4a07-a14c-caf84f9a4abd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:19.849884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c800a74-f433-478b-87f5-dc8c19346fab None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:19.860435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 532/1082] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:49:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:19.875818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:19.893395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7daf8fc-227b-4dc1-b005-8dc0fe5dd2fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:19.894063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf8fc-227b-4dc1-b005-8dc0fe5dd2fe None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:19.894454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf8fc-227b-4dc1-b005-8dc0fe5dd2fe None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:19.894724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7daf8fc-227b-4dc1-b005-8dc0fe5dd2fe None admin] PATH_INFO: `/v3/projects/5f4904a204af495ea615f536dcc5268b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:19.895924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daf8fc-227b-4dc1-b005-8dc0fe5dd2fe None admin] RBAC: Authorizing `identity:get_project(project_id=5f4904a204af495ea615f536dcc5268b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:19.897463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7daf8fc-227b-4dc1-b005-8dc0fe5dd2fe None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:19.899391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 551/1083] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:49:19 2022] GET /identity/v3/projects/5f4904a204af495ea615f536dcc5268b => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:19.905509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524892e-a5ef-4a07-a14c-caf84f9a4abd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:19.942734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:20.019929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524892e-a5ef-4a07-a14c-caf84f9a4abd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:20.057359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e84bc64-847c-4e8b-be5d-3d0f288b8f73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:20.058423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84bc64-847c-4e8b-be5d-3d0f288b8f73 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:20.058703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84bc64-847c-4e8b-be5d-3d0f288b8f73 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:20.058970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e84bc64-847c-4e8b-be5d-3d0f288b8f73 None admin] PATH_INFO: `/v3/projects/5f4904a204af495ea615f536dcc5268b/users/5dee8b64cc58448cab1a78b1d53ca562/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:20.060937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84bc64-847c-4e8b-be5d-3d0f288b8f73 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f4904a204af495ea615f536dcc5268b, user_id=5dee8b64cc58448cab1a78b1d53ca562, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:20.062464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e84bc64-847c-4e8b-be5d-3d0f288b8f73 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:20.080355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 533/1084] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:49:19 2022] PUT /identity/v3/projects/5f4904a204af495ea615f536dcc5268b/users/5dee8b64cc58448cab1a78b1d53ca562/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:49:21.421993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a635c8-91e3-4fa4-b0a3-e68ed58e79f6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:21.422665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a635c8-91e3-4fa4-b0a3-e68ed58e79f6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:21.422665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a635c8-91e3-4fa4-b0a3-e68ed58e79f6 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:21.423885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 552/1085] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:21.429389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261ae36-8e7f-4d61-bd00-d6658c70a3be None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:21.429819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261ae36-8e7f-4d61-bd00-d6658c70a3be None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:21.429922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3261ae36-8e7f-4d61-bd00-d6658c70a3be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:21.448825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3261ae36-8e7f-4d61-bd00-d6658c70a3be None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:21.449993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3261ae36-8e7f-4d61-bd00-d6658c70a3be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:21.517233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 534/1086] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:21.787097 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a635c8-91e3-4fa4-b0a3-e68ed58e79f6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:21.897703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a635c8-91e3-4fa4-b0a3-e68ed58e79f6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:21.935374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f82e2-838c-4f0c-97a0-fb567526747d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:21.936012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f82e2-838c-4f0c-97a0-fb567526747d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:21.936291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f82e2-838c-4f0c-97a0-fb567526747d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:21.936569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849f82e2-838c-4f0c-97a0-fb567526747d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:21.947269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849f82e2-838c-4f0c-97a0-fb567526747d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-849f82e2-838c-4f0c-97a0-fb567526747d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:21.952279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:21.954430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:21.955388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 553/1087] 10.222.0.25 () {56 vars in 1120 bytes} [Tue Sep 13 06:49:21 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:21.959440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261ae36-8e7f-4d61-bd00-d6658c70a3be None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:21.984619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:22.064240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3261ae36-8e7f-4d61-bd00-d6658c70a3be None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:22.102186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c542ab3-08d0-4bd7-bdc6-39c4402fc364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:22.102760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542ab3-08d0-4bd7-bdc6-39c4402fc364 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.103043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542ab3-08d0-4bd7-bdc6-39c4402fc364 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.103316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c542ab3-08d0-4bd7-bdc6-39c4402fc364 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.103998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542ab3-08d0-4bd7-bdc6-39c4402fc364 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:22.104399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542ab3-08d0-4bd7-bdc6-39c4402fc364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:22.105844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c542ab3-08d0-4bd7-bdc6-39c4402fc364 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:22.115271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 535/1088] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Sep 13 06:49:21 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:22.121349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f82e2-838c-4f0c-97a0-fb567526747d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:22.172250 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:22.202324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31523f7b-de78-414d-b9b6-edbc01fcea6b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.202621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31523f7b-de78-414d-b9b6-edbc01fcea6b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.202869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31523f7b-de78-414d-b9b6-edbc01fcea6b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.204237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 536/1089] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:49:22.209322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37e63b-7f4a-4745-a8d7-87bdf316ee07 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.209608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37e63b-7f4a-4745-a8d7-87bdf316ee07 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.209867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a37e63b-7f4a-4745-a8d7-87bdf316ee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.224514 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849f82e2-838c-4f0c-97a0-fb567526747d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:22.228688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9a37e63b-7f4a-4745-a8d7-87bdf316ee07 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:22.230000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9a37e63b-7f4a-4745-a8d7-87bdf316ee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:22.259334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a46798-d0b1-436c-8e4f-d579cfe30606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:22.259967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a46798-d0b1-436c-8e4f-d579cfe30606 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.260245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a46798-d0b1-436c-8e4f-d579cfe30606 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.260558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a46798-d0b1-436c-8e4f-d579cfe30606 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.270636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a46798-d0b1-436c-8e4f-d579cfe30606 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:22.272107 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a46798-d0b1-436c-8e4f-d579cfe30606 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-b3a46798-d0b1-436c-8e4f-d579cfe30606 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:22.287190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:22.288391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:49:22.294475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:22.294475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:22.294475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:22.294475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 554/1090] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:22.294741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a46798-d0b1-436c-8e4f-d579cfe30606 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:22.299326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 537/1091] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:22.390967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:22.394482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a46798-d0b1-436c-8e4f-d579cfe30606 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:22.429034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283e625-986b-4163-b7f4-1900fa1d53c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:22.429664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283e625-986b-4163-b7f4-1900fa1d53c9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.429903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283e625-986b-4163-b7f4-1900fa1d53c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.430197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5283e625-986b-4163-b7f4-1900fa1d53c9 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.430815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283e625-986b-4163-b7f4-1900fa1d53c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:22.431110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283e625-986b-4163-b7f4-1900fa1d53c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:22.432555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5283e625-986b-4163-b7f4-1900fa1d53c9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:22.444871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 555/1092] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:49:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:22.450951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37e63b-7f4a-4745-a8d7-87bdf316ee07 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:22.467297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:22.553410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a37e63b-7f4a-4745-a8d7-87bdf316ee07 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:22.565400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283e625-986b-4163-b7f4-1900fa1d53c9 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:22.588574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbb111-24b8-4976-a550-54ccf1ed24dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:22.589276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbb111-24b8-4976-a550-54ccf1ed24dd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.589579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbb111-24b8-4976-a550-54ccf1ed24dd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.589850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cbb111-24b8-4976-a550-54ccf1ed24dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.599032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbb111-24b8-4976-a550-54ccf1ed24dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:22.600521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cbb111-24b8-4976-a550-54ccf1ed24dd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-55cbb111-24b8-4976-a550-54ccf1ed24dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:49:22.612521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:22.613629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:22.614852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 538/1093] 10.222.0.25 () {56 vars in 1093 bytes} [Tue Sep 13 06:49:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:22.619441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbb111-24b8-4976-a550-54ccf1ed24dd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:22.666991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5283e625-986b-4163-b7f4-1900fa1d53c9 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:22.700810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e00f0-ccc1-4179-8acb-c4f1eaeaaa9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:22.701452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e00f0-ccc1-4179-8acb-c4f1eaeaaa9f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.701561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:22.701729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e00f0-ccc1-4179-8acb-c4f1eaeaaa9f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.701999 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615e00f0-ccc1-4179-8acb-c4f1eaeaaa9f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.712497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615e00f0-ccc1-4179-8acb-c4f1eaeaaa9f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-615e00f0-ccc1-4179-8acb-c4f1eaeaaa9f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:22.717352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:22.718431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:22.719767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 556/1094] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:49:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:22.724548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e00f0-ccc1-4179-8acb-c4f1eaeaaa9f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:22.728776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cbb111-24b8-4976-a550-54ccf1ed24dd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:22.746040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:22.764949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fef036-8892-416e-9791-9f79048e2c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:22.765563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fef036-8892-416e-9791-9f79048e2c82 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.765843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fef036-8892-416e-9791-9f79048e2c82 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.766131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fef036-8892-416e-9791-9f79048e2c82 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.766870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fef036-8892-416e-9791-9f79048e2c82 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:22.767180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fef036-8892-416e-9791-9f79048e2c82 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:22.768655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fef036-8892-416e-9791-9f79048e2c82 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:22.785740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 539/1095] 10.222.0.25 () {56 vars in 1133 bytes} [Tue Sep 13 06:49:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:22.791570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fef036-8892-416e-9791-9f79048e2c82 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:22.827099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615e00f0-ccc1-4179-8acb-c4f1eaeaaa9f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:22.861523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323667c-88c3-48a0-b0d4-2d48180f0414 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:22.862156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323667c-88c3-48a0-b0d4-2d48180f0414 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.862433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323667c-88c3-48a0-b0d4-2d48180f0414 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.862723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6323667c-88c3-48a0-b0d4-2d48180f0414 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.863373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323667c-88c3-48a0-b0d4-2d48180f0414 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:22.863676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323667c-88c3-48a0-b0d4-2d48180f0414 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:22.865063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6323667c-88c3-48a0-b0d4-2d48180f0414 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:22.873621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 557/1096] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:49:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:22.881595 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323667c-88c3-48a0-b0d4-2d48180f0414 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:22.892643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fef036-8892-416e-9791-9f79048e2c82 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:22.921628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:22.928471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829b268-be1b-4eb0-b4b5-70c655fc9fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:22.929086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829b268-be1b-4eb0-b4b5-70c655fc9fdc None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:22.929366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829b268-be1b-4eb0-b4b5-70c655fc9fdc None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:22.929636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829b268-be1b-4eb0-b4b5-70c655fc9fdc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:22.962765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:22.982067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6323667c-88c3-48a0-b0d4-2d48180f0414 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:22.997264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829b268-be1b-4eb0-b4b5-70c655fc9fdc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:22.998791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829b268-be1b-4eb0-b4b5-70c655fc9fdc None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.016914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0c2d8-9a8b-4ea9-8319-12cda56f5bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:23.017550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0c2d8-9a8b-4ea9-8319-12cda56f5bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:23.017828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0c2d8-9a8b-4ea9-8319-12cda56f5bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:23.018098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f0c2d8-9a8b-4ea9-8319-12cda56f5bc7 None admin] PATH_INFO: `/v3/users/5dee8b64cc58448cab1a78b1d53ca562` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:23.019242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0c2d8-9a8b-4ea9-8319-12cda56f5bc7 None admin] RBAC: Authorizing `identity:get_user(user_id=5dee8b64cc58448cab1a78b1d53ca562)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:23.020680 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f0c2d8-9a8b-4ea9-8319-12cda56f5bc7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.022326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 558/1097] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:22 2022] GET /identity/v3/users/5dee8b64cc58448cab1a78b1d53ca562 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:23.028136 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0c2d8-9a8b-4ea9-8319-12cda56f5bc7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:23.098037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 540/1098] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:22 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 06:49:23.103840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829b268-be1b-4eb0-b4b5-70c655fc9fdc None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:23.126296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f0c2d8-9a8b-4ea9-8319-12cda56f5bc7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:23.133631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:23.159754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9064a-13af-4ba3-abc5-476a77b58a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:23.160365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9064a-13af-4ba3-abc5-476a77b58a41 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:23.160643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9064a-13af-4ba3-abc5-476a77b58a41 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:23.160915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb9064a-13af-4ba3-abc5-476a77b58a41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:23.179093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:23.204809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829b268-be1b-4eb0-b4b5-70c655fc9fdc None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:23.232297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9064a-13af-4ba3-abc5-476a77b58a41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:23.233766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb9064a-13af-4ba3-abc5-476a77b58a41 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.240665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9ce23-a303-498c-9096-b81791f0888b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:23.241297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9ce23-a303-498c-9096-b81791f0888b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:23.241572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9ce23-a303-498c-9096-b81791f0888b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:23.241838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d9ce23-a303-498c-9096-b81791f0888b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:23.251904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9ce23-a303-498c-9096-b81791f0888b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:23.253333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d9ce23-a303-498c-9096-b81791f0888b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e4d9ce23-a303-498c-9096-b81791f0888b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:23.269999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:23.271160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:49:23.272102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:23.272102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:49:23.272102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:23.272102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 541/1099] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:23.277279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9ce23-a303-498c-9096-b81791f0888b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:23.340403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 559/1100] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 13 06:49:23.346350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9064a-13af-4ba3-abc5-476a77b58a41 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:23.367221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:23.383825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d9ce23-a303-498c-9096-b81791f0888b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:23.395341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:23.417614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cabf4-3bf4-43ca-a6f7-20fd4ed6b014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:23.418258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cabf4-3bf4-43ca-a6f7-20fd4ed6b014 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:23.418543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cabf4-3bf4-43ca-a6f7-20fd4ed6b014 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:23.418815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cabf4-3bf4-43ca-a6f7-20fd4ed6b014 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:23.419447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cabf4-3bf4-43ca-a6f7-20fd4ed6b014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:23.419750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cabf4-3bf4-43ca-a6f7-20fd4ed6b014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:23.421129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cabf4-3bf4-43ca-a6f7-20fd4ed6b014 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.434594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 542/1101] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:49:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:23.440202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cabf4-3bf4-43ca-a6f7-20fd4ed6b014 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:23.448568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb9064a-13af-4ba3-abc5-476a77b58a41 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:23.483048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61682a-4c01-4090-8a92-1e8e478dff0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:23.483682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61682a-4c01-4090-8a92-1e8e478dff0b None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:23.483956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61682a-4c01-4090-8a92-1e8e478dff0b None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:23.484224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de61682a-4c01-4090-8a92-1e8e478dff0b None admin] PATH_INFO: `/v3/projects/5f4904a204af495ea615f536dcc5268b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:23.485318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61682a-4c01-4090-8a92-1e8e478dff0b None admin] RBAC: Authorizing `identity:get_project(project_id=5f4904a204af495ea615f536dcc5268b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:23.486797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de61682a-4c01-4090-8a92-1e8e478dff0b None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.488415 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 560/1102] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:49:23 2022] GET /identity/v3/projects/5f4904a204af495ea615f536dcc5268b => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:23.494574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61682a-4c01-4090-8a92-1e8e478dff0b None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:23.541059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cabf4-3bf4-43ca-a6f7-20fd4ed6b014 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:23.574315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6564665-86e0-496b-bc50-93408cc3049e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:23.574939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6564665-86e0-496b-bc50-93408cc3049e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:23.575212 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6564665-86e0-496b-bc50-93408cc3049e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:23.575475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6564665-86e0-496b-bc50-93408cc3049e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:23.582210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6564665-86e0-496b-bc50-93408cc3049e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:23.583696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6564665-86e0-496b-bc50-93408cc3049e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.591881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de61682a-4c01-4090-8a92-1e8e478dff0b None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a6564665-86e0-496b-bc50-93408cc3049e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:49:23.594938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:23.595957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:23.596904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:49:23.596904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:49:23.596904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:49:23.596904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:49:23.596904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:49:23.596904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:49:23.596904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:49:23.596904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:23.601118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 543/1103] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:49:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:23.601890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6564665-86e0-496b-bc50-93408cc3049e None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:23.606356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:23.612963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:23.630813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc3783-72ec-4ae8-9605-bd0444671f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:23.631422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc3783-72ec-4ae8-9605-bd0444671f0d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:23.631679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc3783-72ec-4ae8-9605-bd0444671f0d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:23.631944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cc3783-72ec-4ae8-9605-bd0444671f0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:23.632657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc3783-72ec-4ae8-9605-bd0444671f0d None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=5f4904a204af495ea615f536dcc5268b, user.id=5dee8b64cc58448cab1a78b1d53ca562) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:23.632961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc3783-72ec-4ae8-9605-bd0444671f0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:23.634422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cc3783-72ec-4ae8-9605-bd0444671f0d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.641366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 561/1104] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:49:23 2022] GET /identity/v3/role_assignments?user.id=5dee8b64cc58448cab1a78b1d53ca562&scope.project.id=5f4904a204af495ea615f536dcc5268b&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:23.715326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6564665-86e0-496b-bc50-93408cc3049e None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:23.755357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86085b75-5e95-48bb-9b1b-b99806276511 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:23.755987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86085b75-5e95-48bb-9b1b-b99806276511 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:23.756274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86085b75-5e95-48bb-9b1b-b99806276511 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:23.756542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86085b75-5e95-48bb-9b1b-b99806276511 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:23.757246 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86085b75-5e95-48bb-9b1b-b99806276511 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:23.757555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86085b75-5e95-48bb-9b1b-b99806276511 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:23.759045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86085b75-5e95-48bb-9b1b-b99806276511 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.772710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 544/1105] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:49:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:23.779247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc3783-72ec-4ae8-9605-bd0444671f0d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:23.888232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cc3783-72ec-4ae8-9605-bd0444671f0d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:23.921198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dc7764-0683-495b-b553-103ef9668070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:23.921812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7764-0683-495b-b553-103ef9668070 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:23.922116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7764-0683-495b-b553-103ef9668070 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:23.922387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dc7764-0683-495b-b553-103ef9668070 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:23.923108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc7764-0683-495b-b553-103ef9668070 None admin] RBAC: Adding query filter params (role.id=d0cb30355e064a3d8562073f93479915, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=87f3a9ffb88f4faa85edc3dabce0c8f5) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:23.923418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc7764-0683-495b-b553-103ef9668070 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:23.924850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dc7764-0683-495b-b553-103ef9668070 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:23.930638 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 562/1106] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:49:23 2022] GET /identity/v3/role_assignments?user.id=87f3a9ffb88f4faa85edc3dabce0c8f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=d0cb30355e064a3d8562073f93479915 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:25.833989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e93067-6e58-4e70-ba29-dd8355bbc570 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:25.834953 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e93067-6e58-4e70-ba29-dd8355bbc570 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:25.834953 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e93067-6e58-4e70-ba29-dd8355bbc570 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:25.837096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 545/1107] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:25.843911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be570e71-6151-433e-9853-ab12660b4c7c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:25.844384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be570e71-6151-433e-9853-ab12660b4c7c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:25.844477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be570e71-6151-433e-9853-ab12660b4c7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:25.863875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-be570e71-6151-433e-9853-ab12660b4c7c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:25.864977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-be570e71-6151-433e-9853-ab12660b4c7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:25.912820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:25.939426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 563/1108] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:26.211295 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e93067-6e58-4e70-ba29-dd8355bbc570 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:26.284469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:26.327314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e93067-6e58-4e70-ba29-dd8355bbc570 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:26.370132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddee65e0-edca-4da6-a93b-74ba19da7a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:26.370805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee65e0-edca-4da6-a93b-74ba19da7a5e None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:26.371081 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee65e0-edca-4da6-a93b-74ba19da7a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:26.371354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddee65e0-edca-4da6-a93b-74ba19da7a5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:26.372579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee65e0-edca-4da6-a93b-74ba19da7a5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:26.374023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddee65e0-edca-4da6-a93b-74ba19da7a5e None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:26.375636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 546/1109] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:26.381806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be570e71-6151-433e-9853-ab12660b4c7c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:26.478988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be570e71-6151-433e-9853-ab12660b4c7c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:26.518269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec322a-ce6e-4026-8094-df292d4bf7fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:26.518907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec322a-ce6e-4026-8094-df292d4bf7fc None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:26.519186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec322a-ce6e-4026-8094-df292d4bf7fc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:26.519458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec322a-ce6e-4026-8094-df292d4bf7fc None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:26.520119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec322a-ce6e-4026-8094-df292d4bf7fc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:26.521574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec322a-ce6e-4026-8094-df292d4bf7fc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:26.546597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 564/1110] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:49:26 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:49:28.713966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cce5b-dccc-4a2f-a11f-e5ff67a03b80 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:28.714615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cce5b-dccc-4a2f-a11f-e5ff67a03b80 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:28.714615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703cce5b-dccc-4a2f-a11f-e5ff67a03b80 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:28.715869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 547/1111] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:28.722403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967ea5a-4686-4e14-a917-58d215255279 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:28.722847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967ea5a-4686-4e14-a917-58d215255279 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:28.722952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e967ea5a-4686-4e14-a917-58d215255279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:28.745621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e967ea5a-4686-4e14-a917-58d215255279 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:28.746840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e967ea5a-4686-4e14-a917-58d215255279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:28.775755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:28.825843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 565/1112] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:29.090469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cce5b-dccc-4a2f-a11f-e5ff67a03b80 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:29.200955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703cce5b-dccc-4a2f-a11f-e5ff67a03b80 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:29.230836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:29.237278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0a199-3329-475b-ac32-37893eb69dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:29.237905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0a199-3329-475b-ac32-37893eb69dce None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:29.238219 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0a199-3329-475b-ac32-37893eb69dce None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:29.238491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ec0a199-3329-475b-ac32-37893eb69dce None admin] PATH_INFO: `/v3/roles/c622170611454f18bf01aa658a8cc06e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:29.239893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0a199-3329-475b-ac32-37893eb69dce None admin] RBAC: Authorizing `identity:get_role(role_id=c622170611454f18bf01aa658a8cc06e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:29.241274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ec0a199-3329-475b-ac32-37893eb69dce None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:29.242601 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 548/1113] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:29 2022] GET /identity/v3/roles/c622170611454f18bf01aa658a8cc06e => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:29.249098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967ea5a-4686-4e14-a917-58d215255279 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:29.349448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e967ea5a-4686-4e14-a917-58d215255279 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:29.386369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ae1a4-2f07-4635-8e99-ec66f6a37dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:29.386987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ae1a4-2f07-4635-8e99-ec66f6a37dff None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:29.387260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ae1a4-2f07-4635-8e99-ec66f6a37dff None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:29.387532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2ae1a4-2f07-4635-8e99-ec66f6a37dff None admin] PATH_INFO: `/v3/users/61f8273a5a6f4f06a470f8b23035e303` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:29.403239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ae1a4-2f07-4635-8e99-ec66f6a37dff None admin] RBAC: Authorizing `identity:get_user(user_id=61f8273a5a6f4f06a470f8b23035e303)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:29.404691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2ae1a4-2f07-4635-8e99-ec66f6a37dff None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:29.406307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 566/1114] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:29 2022] GET /identity/v3/users/61f8273a5a6f4f06a470f8b23035e303 => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:29.415232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0a199-3329-475b-ac32-37893eb69dce None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:29.520906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ec0a199-3329-475b-ac32-37893eb69dce None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:29.561293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48553bc5-8daa-4520-9b3c-e22f04fee802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:29.561900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48553bc5-8daa-4520-9b3c-e22f04fee802 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:29.562208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48553bc5-8daa-4520-9b3c-e22f04fee802 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:29.562479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48553bc5-8daa-4520-9b3c-e22f04fee802 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:29.630452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:29.633426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48553bc5-8daa-4520-9b3c-e22f04fee802 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:29.636396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48553bc5-8daa-4520-9b3c-e22f04fee802 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:29.747091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 549/1115] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:49:29.756340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ae1a4-2f07-4635-8e99-ec66f6a37dff None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:29.786826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:29.872451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2ae1a4-2f07-4635-8e99-ec66f6a37dff None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:29.912561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dddf1fab-4833-4d8f-8673-bec5c3ed89c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:29.913474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf1fab-4833-4d8f-8673-bec5c3ed89c3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:29.913863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf1fab-4833-4d8f-8673-bec5c3ed89c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:29.914315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dddf1fab-4833-4d8f-8673-bec5c3ed89c3 None admin] PATH_INFO: `/v3/projects/5f4904a204af495ea615f536dcc5268b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:29.916235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf1fab-4833-4d8f-8673-bec5c3ed89c3 None admin] RBAC: Authorizing `identity:get_project(project_id=5f4904a204af495ea615f536dcc5268b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:29.918738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dddf1fab-4833-4d8f-8673-bec5c3ed89c3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:29.921247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 567/1116] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:49:29 2022] GET /identity/v3/projects/5f4904a204af495ea615f536dcc5268b => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:29.931856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48553bc5-8daa-4520-9b3c-e22f04fee802 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:30.048310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:30.052457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48553bc5-8daa-4520-9b3c-e22f04fee802 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:30.099950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d90f74-5129-49fa-ac87-cd3cf974d5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:30.100556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90f74-5129-49fa-ac87-cd3cf974d5fe None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:30.100837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90f74-5129-49fa-ac87-cd3cf974d5fe None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:30.101112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d90f74-5129-49fa-ac87-cd3cf974d5fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:30.101840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d90f74-5129-49fa-ac87-cd3cf974d5fe None admin] RBAC: Adding query filter params (role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=5f4904a204af495ea615f536dcc5268b, user.id=61f8273a5a6f4f06a470f8b23035e303) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:30.102175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d90f74-5129-49fa-ac87-cd3cf974d5fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:30.103620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d90f74-5129-49fa-ac87-cd3cf974d5fe None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:30.109362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 550/1117] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:49:29 2022] GET /identity/v3/role_assignments?user.id=61f8273a5a6f4f06a470f8b23035e303&scope.project.id=5f4904a204af495ea615f536dcc5268b&role.id=c622170611454f18bf01aa658a8cc06e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:32.219747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359974e-1adc-41f5-840b-ef0490174e87 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:32.220331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359974e-1adc-41f5-840b-ef0490174e87 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:32.220331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6359974e-1adc-41f5-840b-ef0490174e87 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:32.221652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 568/1118] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:32.228096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595289c5-417f-4db3-8dce-089d394dabbf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:32.228837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595289c5-417f-4db3-8dce-089d394dabbf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:32.229279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595289c5-417f-4db3-8dce-089d394dabbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:32.250909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-595289c5-417f-4db3-8dce-089d394dabbf None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:32.252099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-595289c5-417f-4db3-8dce-089d394dabbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:32.331043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 551/1119] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:32.608675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359974e-1adc-41f5-840b-ef0490174e87 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:32.703091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:32.720258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6359974e-1adc-41f5-840b-ef0490174e87 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:32.755976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b7f18-cc6b-4bad-8602-d603ec4bdafd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:32.756599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7f18-cc6b-4bad-8602-d603ec4bdafd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:32.756834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7f18-cc6b-4bad-8602-d603ec4bdafd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:32.757134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4b7f18-cc6b-4bad-8602-d603ec4bdafd None admin] PATH_INFO: `/v3/roles/c622170611454f18bf01aa658a8cc06e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:32.758682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b7f18-cc6b-4bad-8602-d603ec4bdafd None admin] RBAC: Authorizing `identity:get_role(role_id=c622170611454f18bf01aa658a8cc06e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:32.760064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4b7f18-cc6b-4bad-8602-d603ec4bdafd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:32.761605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 569/1120] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:32 2022] GET /identity/v3/roles/c622170611454f18bf01aa658a8cc06e => generated 223 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:32.767778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595289c5-417f-4db3-8dce-089d394dabbf None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:32.872278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:32.880713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595289c5-417f-4db3-8dce-089d394dabbf None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:32.919086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61377a1-e335-47e4-8f42-f930bfb2f406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:32.919978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61377a1-e335-47e4-8f42-f930bfb2f406 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:32.920324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61377a1-e335-47e4-8f42-f930bfb2f406 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:32.920626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61377a1-e335-47e4-8f42-f930bfb2f406 None admin] PATH_INFO: `/v3/users/61f8273a5a6f4f06a470f8b23035e303` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:32.922606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61377a1-e335-47e4-8f42-f930bfb2f406 None admin] RBAC: Authorizing `identity:get_user(user_id=61f8273a5a6f4f06a470f8b23035e303)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:32.924861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61377a1-e335-47e4-8f42-f930bfb2f406 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:32.927318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 552/1121] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:32 2022] GET /identity/v3/users/61f8273a5a6f4f06a470f8b23035e303 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:32.935619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b7f18-cc6b-4bad-8602-d603ec4bdafd None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:33.039467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4b7f18-cc6b-4bad-8602-d603ec4bdafd None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:33.076449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783ca23c-f6ef-429d-a471-a27053943bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:33.077364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ca23c-f6ef-429d-a471-a27053943bfe None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:33.077719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ca23c-f6ef-429d-a471-a27053943bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:33.078110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783ca23c-f6ef-429d-a471-a27053943bfe None admin] PATH_INFO: `/v3/projects/5f4904a204af495ea615f536dcc5268b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:33.080112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ca23c-f6ef-429d-a471-a27053943bfe None admin] RBAC: Authorizing `identity:get_project(project_id=5f4904a204af495ea615f536dcc5268b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:33.082496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783ca23c-f6ef-429d-a471-a27053943bfe None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:33.085048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 570/1122] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:49:32 2022] GET /identity/v3/projects/5f4904a204af495ea615f536dcc5268b => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:33.091345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61377a1-e335-47e4-8f42-f930bfb2f406 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:33.228558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61377a1-e335-47e4-8f42-f930bfb2f406 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:33.264951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175b1d08-a993-4b34-98a3-f93198bdfe13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:33.265606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b1d08-a993-4b34-98a3-f93198bdfe13 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:33.265882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b1d08-a993-4b34-98a3-f93198bdfe13 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:33.266181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175b1d08-a993-4b34-98a3-f93198bdfe13 None admin] PATH_INFO: `/v3/projects/5f4904a204af495ea615f536dcc5268b/users/61f8273a5a6f4f06a470f8b23035e303/roles/c622170611454f18bf01aa658a8cc06e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:33.268646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b1d08-a993-4b34-98a3-f93198bdfe13 None admin] RBAC: Authorizing `identity:create_grant(project_id=5f4904a204af495ea615f536dcc5268b, user_id=61f8273a5a6f4f06a470f8b23035e303, role_id=c622170611454f18bf01aa658a8cc06e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:33.270106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175b1d08-a993-4b34-98a3-f93198bdfe13 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:33.289435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 553/1123] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:49:33 2022] PUT /identity/v3/projects/5f4904a204af495ea615f536dcc5268b/users/61f8273a5a6f4f06a470f8b23035e303/roles/c622170611454f18bf01aa658a8cc06e => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:49:35.399642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bd43c-4a76-4dd8-bc79-98cda2918ed1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:35.400261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bd43c-4a76-4dd8-bc79-98cda2918ed1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:35.400261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2bd43c-4a76-4dd8-bc79-98cda2918ed1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:35.401563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 571/1124] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:35.408001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a168d-d2a3-4e6d-8433-8228ffe23af0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:35.408796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a168d-d2a3-4e6d-8433-8228ffe23af0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:35.408899 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904a168d-d2a3-4e6d-8433-8228ffe23af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:35.432689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-904a168d-d2a3-4e6d-8433-8228ffe23af0 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:35.434280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-904a168d-d2a3-4e6d-8433-8228ffe23af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:35.453117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:35.516073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 554/1125] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:35.792546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bd43c-4a76-4dd8-bc79-98cda2918ed1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:35.820475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:35.902603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2bd43c-4a76-4dd8-bc79-98cda2918ed1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:35.938270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7726a0-a470-496f-a7ea-828516c0aadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:35.938901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7726a0-a470-496f-a7ea-828516c0aadb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:35.939179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7726a0-a470-496f-a7ea-828516c0aadb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:35.939457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7726a0-a470-496f-a7ea-828516c0aadb None admin] PATH_INFO: `/v3/roles/c622170611454f18bf01aa658a8cc06e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:35.940964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7726a0-a470-496f-a7ea-828516c0aadb None admin] RBAC: Authorizing `identity:get_role(role_id=c622170611454f18bf01aa658a8cc06e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:35.942385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7726a0-a470-496f-a7ea-828516c0aadb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:35.943735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 572/1126] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:35 2022] GET /identity/v3/roles/c622170611454f18bf01aa658a8cc06e => generated 223 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:35.950195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a168d-d2a3-4e6d-8433-8228ffe23af0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:36.060846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904a168d-d2a3-4e6d-8433-8228ffe23af0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:36.097205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e556484-76f9-485d-93a1-c4a85dd16c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:36.097829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e556484-76f9-485d-93a1-c4a85dd16c6c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:36.098123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e556484-76f9-485d-93a1-c4a85dd16c6c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:36.098409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e556484-76f9-485d-93a1-c4a85dd16c6c None admin] PATH_INFO: `/v3/users/61f8273a5a6f4f06a470f8b23035e303` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:36.099650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e556484-76f9-485d-93a1-c4a85dd16c6c None admin] RBAC: Authorizing `identity:get_user(user_id=61f8273a5a6f4f06a470f8b23035e303)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:36.101105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e556484-76f9-485d-93a1-c4a85dd16c6c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:36.102711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 555/1127] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:35 2022] GET /identity/v3/users/61f8273a5a6f4f06a470f8b23035e303 => generated 286 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:36.109607 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7726a0-a470-496f-a7ea-828516c0aadb None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:36.223955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:36.234500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7726a0-a470-496f-a7ea-828516c0aadb None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:36.272093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5031635-e52f-47e6-bbf6-430f62e81158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:36.272702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031635-e52f-47e6-bbf6-430f62e81158 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:36.272977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031635-e52f-47e6-bbf6-430f62e81158 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:36.273246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5031635-e52f-47e6-bbf6-430f62e81158 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:36.342616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5031635-e52f-47e6-bbf6-430f62e81158 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:36.344196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5031635-e52f-47e6-bbf6-430f62e81158 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:36.439325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:36.449886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 573/1128] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:49:36.456828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e556484-76f9-485d-93a1-c4a85dd16c6c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:36.481938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:36.564954 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e556484-76f9-485d-93a1-c4a85dd16c6c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:36.597455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a2135d-2d20-40b2-90b4-c040c696f42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:36.598007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2135d-2d20-40b2-90b4-c040c696f42d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:36.598315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2135d-2d20-40b2-90b4-c040c696f42d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:36.598585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a2135d-2d20-40b2-90b4-c040c696f42d None admin] PATH_INFO: `/v3/projects/5f4904a204af495ea615f536dcc5268b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:36.599709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2135d-2d20-40b2-90b4-c040c696f42d None admin] RBAC: Authorizing `identity:get_project(project_id=5f4904a204af495ea615f536dcc5268b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:36.601149 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a2135d-2d20-40b2-90b4-c040c696f42d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:36.602787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 556/1129] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:49:36 2022] GET /identity/v3/projects/5f4904a204af495ea615f536dcc5268b => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:36.609848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5031635-e52f-47e6-bbf6-430f62e81158 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:36.715080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5031635-e52f-47e6-bbf6-430f62e81158 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:36.749736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13700c38-214a-4d76-b4c5-865925570d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:36.750371 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13700c38-214a-4d76-b4c5-865925570d31 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:36.750644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13700c38-214a-4d76-b4c5-865925570d31 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:36.750914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13700c38-214a-4d76-b4c5-865925570d31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:36.751645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13700c38-214a-4d76-b4c5-865925570d31 None admin] RBAC: Adding query filter params (role.id=c622170611454f18bf01aa658a8cc06e, scope.project.id=5f4904a204af495ea615f536dcc5268b, user.id=61f8273a5a6f4f06a470f8b23035e303) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:36.751954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13700c38-214a-4d76-b4c5-865925570d31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:36.753394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13700c38-214a-4d76-b4c5-865925570d31 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:36.759313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 574/1130] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:49:36 2022] GET /identity/v3/role_assignments?user.id=61f8273a5a6f4f06a470f8b23035e303&scope.project.id=5f4904a204af495ea615f536dcc5268b&role.id=c622170611454f18bf01aa658a8cc06e => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:38.889011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7016-2b39-45b9-9968-60e70ce0194b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:38.889622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7016-2b39-45b9-9968-60e70ce0194b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:38.889622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337c7016-2b39-45b9-9968-60e70ce0194b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:38.890999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 557/1131] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:38.898246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0109d6-66dd-40ec-b8cf-21a2b291e6e4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:38.898670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0109d6-66dd-40ec-b8cf-21a2b291e6e4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:38.898796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0109d6-66dd-40ec-b8cf-21a2b291e6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:38.919328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fc0109d6-66dd-40ec-b8cf-21a2b291e6e4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:38.920522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fc0109d6-66dd-40ec-b8cf-21a2b291e6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:38.977027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:38.997320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 575/1132] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:39.268016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7016-2b39-45b9-9968-60e70ce0194b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:39.372087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:39.389825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337c7016-2b39-45b9-9968-60e70ce0194b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:39.431741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9a1d89-1465-49e9-9500-88750f34bd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:39.433282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a1d89-1465-49e9-9500-88750f34bd98 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:39.433561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a1d89-1465-49e9-9500-88750f34bd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:39.433827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9a1d89-1465-49e9-9500-88750f34bd98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:39.435025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a1d89-1465-49e9-9500-88750f34bd98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:39.436475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9a1d89-1465-49e9-9500-88750f34bd98 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:39.438037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 558/1133] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:39.447458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0109d6-66dd-40ec-b8cf-21a2b291e6e4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:39.552300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0109d6-66dd-40ec-b8cf-21a2b291e6e4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:39.602261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc46b11-729d-45c2-988d-ad6bf77bfd62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:39.602890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc46b11-729d-45c2-988d-ad6bf77bfd62 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:39.603172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc46b11-729d-45c2-988d-ad6bf77bfd62 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:39.603444 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc46b11-729d-45c2-988d-ad6bf77bfd62 None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:39.604144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc46b11-729d-45c2-988d-ad6bf77bfd62 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:39.605618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc46b11-729d-45c2-988d-ad6bf77bfd62 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:39.636353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 576/1134] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:49:39 2022] POST /identity/v3/projects => generated 311 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:49:41.805172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc670e7-242c-47ea-b096-e055189464f4 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:41.806144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc670e7-242c-47ea-b096-e055189464f4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:41.806144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc670e7-242c-47ea-b096-e055189464f4 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:41.808202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 559/1135] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:41.814699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8172986-f3a6-4947-8bf2-580b56448830 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:41.815134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8172986-f3a6-4947-8bf2-580b56448830 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:41.815267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8172986-f3a6-4947-8bf2-580b56448830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:41.843465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c8172986-f3a6-4947-8bf2-580b56448830 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:41.844650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c8172986-f3a6-4947-8bf2-580b56448830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:41.874981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:41.925534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 577/1136] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:42.198307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc670e7-242c-47ea-b096-e055189464f4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:42.314700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc670e7-242c-47ea-b096-e055189464f4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:42.354810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be649a-5a26-46de-a0e8-5fd09cb196ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:42.355441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be649a-5a26-46de-a0e8-5fd09cb196ff None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:42.355718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be649a-5a26-46de-a0e8-5fd09cb196ff None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:42.355989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be649a-5a26-46de-a0e8-5fd09cb196ff None admin] PATH_INFO: `/v3/domains/default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:42.357167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be649a-5a26-46de-a0e8-5fd09cb196ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:42.358650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be649a-5a26-46de-a0e8-5fd09cb196ff None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:42.360214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 560/1137] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:49:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:42.366257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8172986-f3a6-4947-8bf2-580b56448830 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:42.497016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8172986-f3a6-4947-8bf2-580b56448830 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:42.543654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74df10b-f6f7-4582-b37b-dabcc8e61d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:42.544262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74df10b-f6f7-4582-b37b-dabcc8e61d15 None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:42.544542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74df10b-f6f7-4582-b37b-dabcc8e61d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:42.544814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74df10b-f6f7-4582-b37b-dabcc8e61d15 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:42.545465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74df10b-f6f7-4582-b37b-dabcc8e61d15 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:42.546942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74df10b-f6f7-4582-b37b-dabcc8e61d15 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:42.577802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 578/1138] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:49:42 2022] POST /identity/v3/users => generated 286 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:49:44.731639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b3d5e-0693-4cb3-b11e-5e25f2f08b3e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:44.732666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b3d5e-0693-4cb3-b11e-5e25f2f08b3e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:44.732666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747b3d5e-0693-4cb3-b11e-5e25f2f08b3e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:44.734701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 561/1139] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:44.743489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dd59b-6965-49d4-9965-a4d1acdba6af None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:44.743915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dd59b-6965-49d4-9965-a4d1acdba6af None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:44.744058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767dd59b-6965-49d4-9965-a4d1acdba6af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:44.763758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-767dd59b-6965-49d4-9965-a4d1acdba6af None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:44.764956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-767dd59b-6965-49d4-9965-a4d1acdba6af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:44.782525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:44.835979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 579/1140] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:45.100169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b3d5e-0693-4cb3-b11e-5e25f2f08b3e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:45.132629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:45.230542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747b3d5e-0693-4cb3-b11e-5e25f2f08b3e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:45.267579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff2378-2175-4933-8dfb-a2127ad50215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:45.268203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff2378-2175-4933-8dfb-a2127ad50215 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:45.268476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff2378-2175-4933-8dfb-a2127ad50215 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:45.268743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff2378-2175-4933-8dfb-a2127ad50215 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:45.280645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff2378-2175-4933-8dfb-a2127ad50215 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-dcff2378-2175-4933-8dfb-a2127ad50215 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:45.285403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:45.286496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:45.287793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 562/1141] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:49:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:45.294865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dd59b-6965-49d4-9965-a4d1acdba6af None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:45.402879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767dd59b-6965-49d4-9965-a4d1acdba6af None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:45.439451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c783448-6bf3-45e0-a74a-60cf5219f4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:45.440070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c783448-6bf3-45e0-a74a-60cf5219f4c5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:45.440346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c783448-6bf3-45e0-a74a-60cf5219f4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:45.440613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c783448-6bf3-45e0-a74a-60cf5219f4c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:45.441282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c783448-6bf3-45e0-a74a-60cf5219f4c5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:45.441585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c783448-6bf3-45e0-a74a-60cf5219f4c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:45.443012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c783448-6bf3-45e0-a74a-60cf5219f4c5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:45.452096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 580/1142] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:49:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:45.458465 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff2378-2175-4933-8dfb-a2127ad50215 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:45.547592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:45.565262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff2378-2175-4933-8dfb-a2127ad50215 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:45.600904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac6448-2f9e-461d-9585-5d0fe9bc8ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:45.601530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac6448-2f9e-461d-9585-5d0fe9bc8ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:45.601817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac6448-2f9e-461d-9585-5d0fe9bc8ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:45.602093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac6448-2f9e-461d-9585-5d0fe9bc8ce3 None admin] PATH_INFO: `/v3/users/3ab392ee990245b9bc3ea672648b0194` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:45.616610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac6448-2f9e-461d-9585-5d0fe9bc8ce3 None admin] RBAC: Authorizing `identity:get_user(user_id=3ab392ee990245b9bc3ea672648b0194)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:45.618103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac6448-2f9e-461d-9585-5d0fe9bc8ce3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:45.620343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 563/1143] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:45 2022] GET /identity/v3/users/3ab392ee990245b9bc3ea672648b0194 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:45.626487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c783448-6bf3-45e0-a74a-60cf5219f4c5 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:45.647280 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:45.733044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c783448-6bf3-45e0-a74a-60cf5219f4c5 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:45.771088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48dca-47f7-40e2-af4f-a9a26e3b9f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:45.771708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48dca-47f7-40e2-af4f-a9a26e3b9f40 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:45.772021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48dca-47f7-40e2-af4f-a9a26e3b9f40 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:45.772373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb48dca-47f7-40e2-af4f-a9a26e3b9f40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:45.849622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb48dca-47f7-40e2-af4f-a9a26e3b9f40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:45.851102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb48dca-47f7-40e2-af4f-a9a26e3b9f40 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:45.868011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:45.953955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 581/1144] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:45 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:49:45.963781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac6448-2f9e-461d-9585-5d0fe9bc8ce3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:46.086561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac6448-2f9e-461d-9585-5d0fe9bc8ce3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:46.121753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c001f49-0b93-43ec-ab57-77ff7d10e999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:46.122396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c001f49-0b93-43ec-ab57-77ff7d10e999 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:46.122668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c001f49-0b93-43ec-ab57-77ff7d10e999 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:46.122940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c001f49-0b93-43ec-ab57-77ff7d10e999 None admin] PATH_INFO: `/v3/projects/3c1d346f2ace467db46e14d515dc6b0a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:46.124181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c001f49-0b93-43ec-ab57-77ff7d10e999 None admin] RBAC: Authorizing `identity:get_project(project_id=3c1d346f2ace467db46e14d515dc6b0a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:46.125624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c001f49-0b93-43ec-ab57-77ff7d10e999 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:46.127249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 564/1145] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:49:45 2022] GET /identity/v3/projects/3c1d346f2ace467db46e14d515dc6b0a => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:46.137040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48dca-47f7-40e2-af4f-a9a26e3b9f40 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:46.249847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb48dca-47f7-40e2-af4f-a9a26e3b9f40 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:46.266736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:46.286437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a298b6-f2aa-4ce5-ab3f-6e042cbad9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:46.287025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a298b6-f2aa-4ce5-ab3f-6e042cbad9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:46.287297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a298b6-f2aa-4ce5-ab3f-6e042cbad9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:46.287562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a298b6-f2aa-4ce5-ab3f-6e042cbad9d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:46.288275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a298b6-f2aa-4ce5-ab3f-6e042cbad9d5 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=3c1d346f2ace467db46e14d515dc6b0a, user.id=3ab392ee990245b9bc3ea672648b0194) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:46.288574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a298b6-f2aa-4ce5-ab3f-6e042cbad9d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:46.290007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a298b6-f2aa-4ce5-ab3f-6e042cbad9d5 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:46.299142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 582/1146] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:49:46 2022] GET /identity/v3/role_assignments?user.id=3ab392ee990245b9bc3ea672648b0194&scope.project.id=3c1d346f2ace467db46e14d515dc6b0a&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:48.397101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33d6ee-3ff3-4ba0-a0bc-72183e59913c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:48.398132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33d6ee-3ff3-4ba0-a0bc-72183e59913c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:48.398132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb33d6ee-3ff3-4ba0-a0bc-72183e59913c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:48.400132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 565/1147] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:48.405704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9179b7-d207-4df4-9373-e1d8f6f0ca1f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:48.406187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9179b7-d207-4df4-9373-e1d8f6f0ca1f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:48.406290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9179b7-d207-4df4-9373-e1d8f6f0ca1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:48.424880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2d9179b7-d207-4df4-9373-e1d8f6f0ca1f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:48.426043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2d9179b7-d207-4df4-9373-e1d8f6f0ca1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:48.500714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 583/1148] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:48.766960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33d6ee-3ff3-4ba0-a0bc-72183e59913c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:48.783217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:48.894633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb33d6ee-3ff3-4ba0-a0bc-72183e59913c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:48.934389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0aec4-97f4-43db-a2f7-8df9393c417a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:48.935013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0aec4-97f4-43db-a2f7-8df9393c417a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:48.935288 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0aec4-97f4-43db-a2f7-8df9393c417a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:48.935559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae0aec4-97f4-43db-a2f7-8df9393c417a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:48.946412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae0aec4-97f4-43db-a2f7-8df9393c417a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-eae0aec4-97f4-43db-a2f7-8df9393c417a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:48.951388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:48.952516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:48.953815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 566/1149] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:49:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:48.960981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9179b7-d207-4df4-9373-e1d8f6f0ca1f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:49.075970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9179b7-d207-4df4-9373-e1d8f6f0ca1f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:49.086628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:49.123961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3bdea-a5c7-4d1f-a09d-dbfe3a8adfe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:49.124576 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3bdea-a5c7-4d1f-a09d-dbfe3a8adfe6 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:49.124851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3bdea-a5c7-4d1f-a09d-dbfe3a8adfe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:49.125114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a3bdea-a5c7-4d1f-a09d-dbfe3a8adfe6 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:49.125775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3bdea-a5c7-4d1f-a09d-dbfe3a8adfe6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:49.126117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3bdea-a5c7-4d1f-a09d-dbfe3a8adfe6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:49.127502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a3bdea-a5c7-4d1f-a09d-dbfe3a8adfe6 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:49.136494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 584/1150] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:49:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:49.143895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0aec4-97f4-43db-a2f7-8df9393c417a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:49.217017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:49.249378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae0aec4-97f4-43db-a2f7-8df9393c417a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:49.284700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3002-def0-44c4-8a8d-97b73ea7ea68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:49.285318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3002-def0-44c4-8a8d-97b73ea7ea68 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:49.285593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3002-def0-44c4-8a8d-97b73ea7ea68 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:49.285868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4a3002-def0-44c4-8a8d-97b73ea7ea68 None admin] PATH_INFO: `/v3/users/3ab392ee990245b9bc3ea672648b0194` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:49.287146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a3002-def0-44c4-8a8d-97b73ea7ea68 None admin] RBAC: Authorizing `identity:get_user(user_id=3ab392ee990245b9bc3ea672648b0194)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:49.288600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4a3002-def0-44c4-8a8d-97b73ea7ea68 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:49.290182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 567/1151] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:49 2022] GET /identity/v3/users/3ab392ee990245b9bc3ea672648b0194 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:49.299672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3bdea-a5c7-4d1f-a09d-dbfe3a8adfe6 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:49.409989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a3bdea-a5c7-4d1f-a09d-dbfe3a8adfe6 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:49.445826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb75d30c-1184-4948-8ea8-36db7fc03ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:49.446481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d30c-1184-4948-8ea8-36db7fc03ab7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:49.446756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d30c-1184-4948-8ea8-36db7fc03ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:49.447026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb75d30c-1184-4948-8ea8-36db7fc03ab7 None admin] PATH_INFO: `/v3/projects/3c1d346f2ace467db46e14d515dc6b0a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:49.448200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d30c-1184-4948-8ea8-36db7fc03ab7 None admin] RBAC: Authorizing `identity:get_project(project_id=3c1d346f2ace467db46e14d515dc6b0a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:49.449648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb75d30c-1184-4948-8ea8-36db7fc03ab7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:49.451310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 585/1152] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:49:49 2022] GET /identity/v3/projects/3c1d346f2ace467db46e14d515dc6b0a => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:49.458126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3002-def0-44c4-8a8d-97b73ea7ea68 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:49.575037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4a3002-def0-44c4-8a8d-97b73ea7ea68 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:49.610200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cba6ec1-c9b5-4d1b-8b1d-03941c2d1357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:49.610853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba6ec1-c9b5-4d1b-8b1d-03941c2d1357 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:49.611135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba6ec1-c9b5-4d1b-8b1d-03941c2d1357 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:49.611408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cba6ec1-c9b5-4d1b-8b1d-03941c2d1357 None admin] PATH_INFO: `/v3/projects/3c1d346f2ace467db46e14d515dc6b0a/users/3ab392ee990245b9bc3ea672648b0194/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:49.613356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba6ec1-c9b5-4d1b-8b1d-03941c2d1357 None admin] RBAC: Authorizing `identity:create_grant(project_id=3c1d346f2ace467db46e14d515dc6b0a, user_id=3ab392ee990245b9bc3ea672648b0194, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:49.614841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cba6ec1-c9b5-4d1b-8b1d-03941c2d1357 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:49.626587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:49.632374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 568/1153] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:49:49 2022] PUT /identity/v3/projects/3c1d346f2ace467db46e14d515dc6b0a/users/3ab392ee990245b9bc3ea672648b0194/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:49:51.715225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9113c-33c4-4ee9-9e25-d8439b3420cd None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:51.715823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9113c-33c4-4ee9-9e25-d8439b3420cd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:51.715823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b9113c-33c4-4ee9-9e25-d8439b3420cd None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:51.717093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 586/1154] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:51.723524 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f05dc-143d-4965-bb8f-55197ce26a83 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:51.724376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f05dc-143d-4965-bb8f-55197ce26a83 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:51.724376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f05dc-143d-4965-bb8f-55197ce26a83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:51.745585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-570f05dc-143d-4965-bb8f-55197ce26a83 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:51.746829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-570f05dc-143d-4965-bb8f-55197ce26a83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:51.817067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 569/1155] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:52.087697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9113c-33c4-4ee9-9e25-d8439b3420cd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:52.136545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:52.205474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b9113c-33c4-4ee9-9e25-d8439b3420cd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:52.239818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405abb0-fb57-4140-a043-f874adc746da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:52.240443 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405abb0-fb57-4140-a043-f874adc746da None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:52.240722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405abb0-fb57-4140-a043-f874adc746da None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:52.240991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9405abb0-fb57-4140-a043-f874adc746da None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:52.252832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9405abb0-fb57-4140-a043-f874adc746da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9405abb0-fb57-4140-a043-f874adc746da None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:49:52.257634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:52.258760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:49:52.259790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 587/1156] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:49:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:49:52.267664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f05dc-143d-4965-bb8f-55197ce26a83 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:52.375941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570f05dc-143d-4965-bb8f-55197ce26a83 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:52.405661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:52.414968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e69bc-5b45-473d-8adc-8af7d6192137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:52.415577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e69bc-5b45-473d-8adc-8af7d6192137 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:52.415853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e69bc-5b45-473d-8adc-8af7d6192137 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:52.416123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737e69bc-5b45-473d-8adc-8af7d6192137 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:52.416828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e69bc-5b45-473d-8adc-8af7d6192137 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:52.417133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e69bc-5b45-473d-8adc-8af7d6192137 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:52.418537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737e69bc-5b45-473d-8adc-8af7d6192137 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:52.426744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 570/1157] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:49:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:52.433435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405abb0-fb57-4140-a043-f874adc746da None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:52.533865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:52.536840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9405abb0-fb57-4140-a043-f874adc746da None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:52.572036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b971-657b-4372-89fe-71a08c4faa49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:52.572741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b971-657b-4372-89fe-71a08c4faa49 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:52.573021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b971-657b-4372-89fe-71a08c4faa49 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:52.573294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad21b971-657b-4372-89fe-71a08c4faa49 None admin] PATH_INFO: `/v3/users/3ab392ee990245b9bc3ea672648b0194` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:52.574474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21b971-657b-4372-89fe-71a08c4faa49 None admin] RBAC: Authorizing `identity:get_user(user_id=3ab392ee990245b9bc3ea672648b0194)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:52.575924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad21b971-657b-4372-89fe-71a08c4faa49 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:52.577498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 588/1158] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:49:52 2022] GET /identity/v3/users/3ab392ee990245b9bc3ea672648b0194 => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:49:52.584235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e69bc-5b45-473d-8adc-8af7d6192137 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:52.697343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737e69bc-5b45-473d-8adc-8af7d6192137 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:52.732481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a5d4-a82b-4790-9c8d-50213d9b65ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:52.733124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a5d4-a82b-4790-9c8d-50213d9b65ec None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:52.733347 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a5d4-a82b-4790-9c8d-50213d9b65ec None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:52.733621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d78a5d4-a82b-4790-9c8d-50213d9b65ec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:52.798645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:52.813536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78a5d4-a82b-4790-9c8d-50213d9b65ec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:52.815048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d78a5d4-a82b-4790-9c8d-50213d9b65ec None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:52.921880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 571/1159] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:49:52.928202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b971-657b-4372-89fe-71a08c4faa49 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:53.049299 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad21b971-657b-4372-89fe-71a08c4faa49 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:53.091339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82dfeec7-99f1-4fc2-8bdd-1aebc5627d10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:53.091973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfeec7-99f1-4fc2-8bdd-1aebc5627d10 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:53.092252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfeec7-99f1-4fc2-8bdd-1aebc5627d10 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:53.092532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82dfeec7-99f1-4fc2-8bdd-1aebc5627d10 None admin] PATH_INFO: `/v3/projects/3c1d346f2ace467db46e14d515dc6b0a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:53.093721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfeec7-99f1-4fc2-8bdd-1aebc5627d10 None admin] RBAC: Authorizing `identity:get_project(project_id=3c1d346f2ace467db46e14d515dc6b0a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:53.095211 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82dfeec7-99f1-4fc2-8bdd-1aebc5627d10 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:53.096854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 589/1160] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:49:52 2022] GET /identity/v3/projects/3c1d346f2ace467db46e14d515dc6b0a => generated 311 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:53.106615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a5d4-a82b-4790-9c8d-50213d9b65ec None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:53.209889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:53.220639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d78a5d4-a82b-4790-9c8d-50213d9b65ec None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:53.255889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdda51-522e-4f31-9094-31e086371837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:53.256497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdda51-522e-4f31-9094-31e086371837 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:53.256778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdda51-522e-4f31-9094-31e086371837 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:53.257051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdda51-522e-4f31-9094-31e086371837 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:53.257780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdda51-522e-4f31-9094-31e086371837 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=3c1d346f2ace467db46e14d515dc6b0a, user.id=3ab392ee990245b9bc3ea672648b0194) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:53.258120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdda51-522e-4f31-9094-31e086371837 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:53.259575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdda51-522e-4f31-9094-31e086371837 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:53.265687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 572/1161] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:49:53 2022] GET /identity/v3/role_assignments?user.id=3ab392ee990245b9bc3ea672648b0194&scope.project.id=3c1d346f2ace467db46e14d515dc6b0a&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:55.471183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3688-8b05-4cc8-8f18-a3512e4a1149 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:55.472076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3688-8b05-4cc8-8f18-a3512e4a1149 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:55.472076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2c3688-8b05-4cc8-8f18-a3512e4a1149 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:55.474226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 590/1162] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:55.481820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b6204-575a-4b60-8a2e-e73b002d4639 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:55.482345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b6204-575a-4b60-8a2e-e73b002d4639 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:55.482345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56b6204-575a-4b60-8a2e-e73b002d4639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:55.502652 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c56b6204-575a-4b60-8a2e-e73b002d4639 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:55.503835 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c56b6204-575a-4b60-8a2e-e73b002d4639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:55.580662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 573/1163] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:49:55.846761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3688-8b05-4cc8-8f18-a3512e4a1149 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:55.888807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:55.962257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2c3688-8b05-4cc8-8f18-a3512e4a1149 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:56.000050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa299f0-4742-4730-aed1-74cfed796e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:56.000651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa299f0-4742-4730-aed1-74cfed796e91 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:56.000930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa299f0-4742-4730-aed1-74cfed796e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:56.001203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa299f0-4742-4730-aed1-74cfed796e91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:56.070023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa299f0-4742-4730-aed1-74cfed796e91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:56.071517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa299f0-4742-4730-aed1-74cfed796e91 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:56.107422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:56.177372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 591/1164] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:49:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:49:56.185692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b6204-575a-4b60-8a2e-e73b002d4639 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:56.281607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:56.304194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56b6204-575a-4b60-8a2e-e73b002d4639 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:56.347506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2509e94d-f8fb-4606-9927-2c657a490be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:56.348388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509e94d-f8fb-4606-9927-2c657a490be7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:56.348670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509e94d-f8fb-4606-9927-2c657a490be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:56.348940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2509e94d-f8fb-4606-9927-2c657a490be7 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:56.356819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509e94d-f8fb-4606-9927-2c657a490be7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:56.358368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2509e94d-f8fb-4606-9927-2c657a490be7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-2509e94d-f8fb-4606-9927-2c657a490be7 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:49:56.370249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:49:56.371956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:49:56.373055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 574/1165] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:49:56 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:49:56.377714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa299f0-4742-4730-aed1-74cfed796e91 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:56.484189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa299f0-4742-4730-aed1-74cfed796e91 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:56.518264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8a1a8-3528-4bea-8e2d-53811fea24ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:56.518864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8a1a8-3528-4bea-8e2d-53811fea24ab None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:56.519143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8a1a8-3528-4bea-8e2d-53811fea24ab None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:56.519416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8a1a8-3528-4bea-8e2d-53811fea24ab None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:56.520056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8a1a8-3528-4bea-8e2d-53811fea24ab None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:49:56.520356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8a1a8-3528-4bea-8e2d-53811fea24ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:56.521732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8a1a8-3528-4bea-8e2d-53811fea24ab None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:56.524064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:49:56.529687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 592/1166] 10.222.0.25 () {56 vars in 1115 bytes} [Tue Sep 13 06:49:56 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:49:58.658171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b3d89-b50c-4325-a266-b9a81b173200 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:58.658717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b3d89-b50c-4325-a266-b9a81b173200 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:58.658717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4b3d89-b50c-4325-a266-b9a81b173200 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:58.660044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 575/1167] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:49:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:49:58.665517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93fe3f-ed4f-4f94-8559-cc1e461440f7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:58.665997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93fe3f-ed4f-4f94-8559-cc1e461440f7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:58.666122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae93fe3f-ed4f-4f94-8559-cc1e461440f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:58.685606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ae93fe3f-ed4f-4f94-8559-cc1e461440f7 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:49:58.686872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ae93fe3f-ed4f-4f94-8559-cc1e461440f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:49:58.756503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 593/1168] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:49:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:49:59.021326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b3d89-b50c-4325-a266-b9a81b173200 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:49:59.128112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4b3d89-b50c-4325-a266-b9a81b173200 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:49:59.164261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d88b3c-5a23-4825-99f7-d7bf6c67c12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:49:59.164850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d88b3c-5a23-4825-99f7-d7bf6c67c12f None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:49:59.165126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d88b3c-5a23-4825-99f7-d7bf6c67c12f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:49:59.165394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d88b3c-5a23-4825-99f7-d7bf6c67c12f None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:49:59.166054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d88b3c-5a23-4825-99f7-d7bf6c67c12f None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:49:59.167453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d88b3c-5a23-4825-99f7-d7bf6c67c12f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:49:59.187277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 576/1169] 10.222.0.25 () {60 vars in 1138 bytes} [Tue Sep 13 06:49:59 2022] POST /identity/v3/domains => generated 267 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:50:01.328706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19699f18-5c66-43e2-b0dd-858d5b1525d7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:01.329625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19699f18-5c66-43e2-b0dd-858d5b1525d7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:01.329625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19699f18-5c66-43e2-b0dd-858d5b1525d7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:01.331781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 594/1170] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:50:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:50:01.340459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f62c5-494b-4263-95a2-06c1e3a33332 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:01.340933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f62c5-494b-4263-95a2-06c1e3a33332 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:01.341027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287f62c5-494b-4263-95a2-06c1e3a33332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:01.353162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:01.365221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-287f62c5-494b-4263-95a2-06c1e3a33332 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:50:01.366422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-287f62c5-494b-4263-95a2-06c1e3a33332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:50:01.434867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 577/1171] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:50:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:50:01.706432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19699f18-5c66-43e2-b0dd-858d5b1525d7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:01.824585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19699f18-5c66-43e2-b0dd-858d5b1525d7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:01.861616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:01.869006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216ae1c9-8eab-460e-9190-e57c8f5cd4b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:01.869653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ae1c9-8eab-460e-9190-e57c8f5cd4b4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:01.869933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ae1c9-8eab-460e-9190-e57c8f5cd4b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:01.870230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216ae1c9-8eab-460e-9190-e57c8f5cd4b4 None admin] PATH_INFO: `/v3/domains/feaf30c392454249bbb2765987e412d9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:01.881720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ae1c9-8eab-460e-9190-e57c8f5cd4b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=feaf30c392454249bbb2765987e412d9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:01.883244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216ae1c9-8eab-460e-9190-e57c8f5cd4b4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:01.884900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 595/1172] 10.222.0.25 () {56 vars in 1183 bytes} [Tue Sep 13 06:50:01 2022] GET /identity/v3/domains/feaf30c392454249bbb2765987e412d9 => generated 267 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:01.891294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f62c5-494b-4263-95a2-06c1e3a33332 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:02.005200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287f62c5-494b-4263-95a2-06c1e3a33332 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:02.041028 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f61e641-1a40-4629-ba40-c132150101b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:02.041628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61e641-1a40-4629-ba40-c132150101b8 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:02.041906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61e641-1a40-4629-ba40-c132150101b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:02.042205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f61e641-1a40-4629-ba40-c132150101b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:02.043055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61e641-1a40-4629-ba40-c132150101b8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:02.044464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f61e641-1a40-4629-ba40-c132150101b8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:02.049245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:02.076426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 578/1173] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:50:01 2022] POST /identity/v3/projects => generated 361 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 06:50:04.336501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b88875-7d3b-4750-ac6e-6f94affad95c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:04.337085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b88875-7d3b-4750-ac6e-6f94affad95c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:04.337085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b88875-7d3b-4750-ac6e-6f94affad95c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:04.339312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 596/1174] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:50:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:50:04.345298 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42826f-4a26-4e25-adc3-ad2a70bf4880 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:04.345780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42826f-4a26-4e25-adc3-ad2a70bf4880 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:04.345874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b42826f-4a26-4e25-adc3-ad2a70bf4880 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:04.365286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7b42826f-4a26-4e25-adc3-ad2a70bf4880 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:50:04.366520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7b42826f-4a26-4e25-adc3-ad2a70bf4880 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:50:04.436874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 579/1175] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:50:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:50:04.708942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b88875-7d3b-4750-ac6e-6f94affad95c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:04.818986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b88875-7d3b-4750-ac6e-6f94affad95c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:04.853568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a8d756-f75d-49de-b3c3-4202f46c26bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:04.854217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8d756-f75d-49de-b3c3-4202f46c26bd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:04.854495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8d756-f75d-49de-b3c3-4202f46c26bd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:04.854768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a8d756-f75d-49de-b3c3-4202f46c26bd None admin] PATH_INFO: `/v3/domains/feaf30c392454249bbb2765987e412d9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:04.856087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8d756-f75d-49de-b3c3-4202f46c26bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=feaf30c392454249bbb2765987e412d9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:04.857534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a8d756-f75d-49de-b3c3-4202f46c26bd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:04.859147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 597/1176] 10.222.0.25 () {56 vars in 1183 bytes} [Tue Sep 13 06:50:04 2022] GET /identity/v3/domains/feaf30c392454249bbb2765987e412d9 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:04.865274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42826f-4a26-4e25-adc3-ad2a70bf4880 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:04.971763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b42826f-4a26-4e25-adc3-ad2a70bf4880 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:04.988262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:05.007558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dbb701-dd8b-4cd7-92c1-ffed497457a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:05.008164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbb701-dd8b-4cd7-92c1-ffed497457a2 None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:05.008436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbb701-dd8b-4cd7-92c1-ffed497457a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:05.008703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dbb701-dd8b-4cd7-92c1-ffed497457a2 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:05.009347 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbb701-dd8b-4cd7-92c1-ffed497457a2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:05.010829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dbb701-dd8b-4cd7-92c1-ffed497457a2 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:05.034049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 580/1177] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:50:04 2022] POST /identity/v3/users => generated 311 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:50:07.221105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7fc38-20bd-4d83-a063-af5b3691a15f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:07.221689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7fc38-20bd-4d83-a063-af5b3691a15f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:07.221689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7fc38-20bd-4d83-a063-af5b3691a15f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:07.223171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 598/1178] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:50:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:50:07.229762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e5133-e24c-4eec-8798-8f129bec94fd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:07.230576 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e5133-e24c-4eec-8798-8f129bec94fd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:07.230963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e5133-e24c-4eec-8798-8f129bec94fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:07.256923 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-020e5133-e24c-4eec-8798-8f129bec94fd None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:50:07.258124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-020e5133-e24c-4eec-8798-8f129bec94fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:50:07.336728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 581/1179] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:50:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:50:07.607049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7fc38-20bd-4d83-a063-af5b3691a15f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:07.671574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:07.718808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7fc38-20bd-4d83-a063-af5b3691a15f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:07.755863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b03ce-6d5a-4384-a582-eeb49b99ef33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:07.756495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b03ce-6d5a-4384-a582-eeb49b99ef33 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:07.756771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b03ce-6d5a-4384-a582-eeb49b99ef33 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:07.757045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b03ce-6d5a-4384-a582-eeb49b99ef33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:07.768712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1b03ce-6d5a-4384-a582-eeb49b99ef33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2e1b03ce-6d5a-4384-a582-eeb49b99ef33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:50:07.773500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:50:07.774623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:50:07.775564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 599/1180] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:50:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:50:07.781090 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e5133-e24c-4eec-8798-8f129bec94fd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:07.886210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e5133-e24c-4eec-8798-8f129bec94fd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:07.887824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:07.925188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4abcc8-7486-47da-b27e-f3c12792cac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:07.925796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4abcc8-7486-47da-b27e-f3c12792cac3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:07.926074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4abcc8-7486-47da-b27e-f3c12792cac3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:07.926358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4abcc8-7486-47da-b27e-f3c12792cac3 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:07.927040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4abcc8-7486-47da-b27e-f3c12792cac3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:50:07.927340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4abcc8-7486-47da-b27e-f3c12792cac3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:07.928714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4abcc8-7486-47da-b27e-f3c12792cac3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:07.937233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 582/1181] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:50:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:07.946384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b03ce-6d5a-4384-a582-eeb49b99ef33 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:08.055115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1b03ce-6d5a-4384-a582-eeb49b99ef33 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:08.072885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:08.092120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a6417-5cca-4489-acbf-72b93def2667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:08.092746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a6417-5cca-4489-acbf-72b93def2667 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:08.093022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a6417-5cca-4489-acbf-72b93def2667 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:08.093295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a6417-5cca-4489-acbf-72b93def2667 None admin] PATH_INFO: `/v3/users/4ac0e9c24af24b50a3d3731fa6cac7dd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:08.106780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a6417-5cca-4489-acbf-72b93def2667 None admin] RBAC: Authorizing `identity:get_user(user_id=4ac0e9c24af24b50a3d3731fa6cac7dd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:08.108241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6a6417-5cca-4489-acbf-72b93def2667 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:08.109840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 600/1182] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:50:07 2022] GET /identity/v3/users/4ac0e9c24af24b50a3d3731fa6cac7dd => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:08.116034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4abcc8-7486-47da-b27e-f3c12792cac3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:08.226291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4abcc8-7486-47da-b27e-f3c12792cac3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:08.264027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aea99-db2b-4e69-b0c4-12d2cf91c0c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:08.264660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aea99-db2b-4e69-b0c4-12d2cf91c0c4 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:08.264869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aea99-db2b-4e69-b0c4-12d2cf91c0c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:08.265167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391aea99-db2b-4e69-b0c4-12d2cf91c0c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:08.296910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:08.337887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aea99-db2b-4e69-b0c4-12d2cf91c0c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:08.339518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391aea99-db2b-4e69-b0c4-12d2cf91c0c4 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:08.446514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 583/1183] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:50:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:50:08.453554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a6417-5cca-4489-acbf-72b93def2667 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:08.556930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a6417-5cca-4489-acbf-72b93def2667 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:08.593527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6548732d-45e6-4fa8-b4d9-7f668309dc82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:08.594174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548732d-45e6-4fa8-b4d9-7f668309dc82 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:08.594578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548732d-45e6-4fa8-b4d9-7f668309dc82 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:08.594855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6548732d-45e6-4fa8-b4d9-7f668309dc82 None admin] PATH_INFO: `/v3/projects/1a756cc77b5f4cbebd3a0b36cb19bf9a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:08.596025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548732d-45e6-4fa8-b4d9-7f668309dc82 None admin] RBAC: Authorizing `identity:get_project(project_id=1a756cc77b5f4cbebd3a0b36cb19bf9a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:08.597477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6548732d-45e6-4fa8-b4d9-7f668309dc82 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:08.599167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 601/1184] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:50:08 2022] GET /identity/v3/projects/1a756cc77b5f4cbebd3a0b36cb19bf9a => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:08.607833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aea99-db2b-4e69-b0c4-12d2cf91c0c4 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:08.676559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:08.719038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391aea99-db2b-4e69-b0c4-12d2cf91c0c4 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:08.757784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae50e261-d2e3-4fde-a6ea-8b8a8278e923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:08.758426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50e261-d2e3-4fde-a6ea-8b8a8278e923 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:08.758711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50e261-d2e3-4fde-a6ea-8b8a8278e923 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:08.758985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae50e261-d2e3-4fde-a6ea-8b8a8278e923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:08.759710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50e261-d2e3-4fde-a6ea-8b8a8278e923 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=1a756cc77b5f4cbebd3a0b36cb19bf9a, user.id=4ac0e9c24af24b50a3d3731fa6cac7dd) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:50:08.760023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50e261-d2e3-4fde-a6ea-8b8a8278e923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:08.761469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae50e261-d2e3-4fde-a6ea-8b8a8278e923 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:08.767410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 584/1185] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:50:08 2022] GET /identity/v3/role_assignments?user.id=4ac0e9c24af24b50a3d3731fa6cac7dd&scope.project.id=1a756cc77b5f4cbebd3a0b36cb19bf9a&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:10.880804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d999a69-55b9-4913-b65c-5173eb01ec1e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:10.881355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d999a69-55b9-4913-b65c-5173eb01ec1e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:10.881355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d999a69-55b9-4913-b65c-5173eb01ec1e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:10.882667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 602/1186] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:50:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:50:10.889163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f98f18-f1de-4d21-8503-82f56019798b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:10.889914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f98f18-f1de-4d21-8503-82f56019798b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:10.890065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f98f18-f1de-4d21-8503-82f56019798b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:10.915052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-88f98f18-f1de-4d21-8503-82f56019798b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:50:10.916252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-88f98f18-f1de-4d21-8503-82f56019798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:50:10.988158 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 585/1187] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:50:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:50:11.256630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d999a69-55b9-4913-b65c-5173eb01ec1e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:11.310292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:11.367451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d999a69-55b9-4913-b65c-5173eb01ec1e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:11.403050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173d31e-e0c8-4610-ab28-139c68ed02b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:11.403671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173d31e-e0c8-4610-ab28-139c68ed02b4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:11.403947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173d31e-e0c8-4610-ab28-139c68ed02b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:11.404232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4173d31e-e0c8-4610-ab28-139c68ed02b4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:11.415553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4173d31e-e0c8-4610-ab28-139c68ed02b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4173d31e-e0c8-4610-ab28-139c68ed02b4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:50:11.420481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:50:11.421583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:50:11.422558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:50:11.426310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 603/1188] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:50:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:50:11.428196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f98f18-f1de-4d21-8503-82f56019798b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:11.491991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:11.543107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f98f18-f1de-4d21-8503-82f56019798b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:11.579216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a3173-4583-4146-88e5-295716c4ca1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:11.579823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a3173-4583-4146-88e5-295716c4ca1b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:11.580101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a3173-4583-4146-88e5-295716c4ca1b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:11.580366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782a3173-4583-4146-88e5-295716c4ca1b None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:11.581025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a3173-4583-4146-88e5-295716c4ca1b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:50:11.581334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a3173-4583-4146-88e5-295716c4ca1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:11.582740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782a3173-4583-4146-88e5-295716c4ca1b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:11.590905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 586/1189] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:50:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:11.597062 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173d31e-e0c8-4610-ab28-139c68ed02b4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:11.702651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4173d31e-e0c8-4610-ab28-139c68ed02b4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:11.705040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:11.740748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036784c-5124-46b4-9524-baa5d7bf6c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:11.741370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036784c-5124-46b4-9524-baa5d7bf6c6e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:11.741644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036784c-5124-46b4-9524-baa5d7bf6c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:11.741911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3036784c-5124-46b4-9524-baa5d7bf6c6e None admin] PATH_INFO: `/v3/users/4ac0e9c24af24b50a3d3731fa6cac7dd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:11.743092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036784c-5124-46b4-9524-baa5d7bf6c6e None admin] RBAC: Authorizing `identity:get_user(user_id=4ac0e9c24af24b50a3d3731fa6cac7dd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:11.744525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3036784c-5124-46b4-9524-baa5d7bf6c6e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:11.746156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 604/1190] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:50:11 2022] GET /identity/v3/users/4ac0e9c24af24b50a3d3731fa6cac7dd => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:11.755649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a3173-4583-4146-88e5-295716c4ca1b None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:11.868406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782a3173-4583-4146-88e5-295716c4ca1b None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:11.888264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:11.905206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5aaec4-9f69-48b1-bb83-7392c16517d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:11.906183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aaec4-9f69-48b1-bb83-7392c16517d8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:11.906544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aaec4-9f69-48b1-bb83-7392c16517d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:11.906898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5aaec4-9f69-48b1-bb83-7392c16517d8 None admin] PATH_INFO: `/v3/projects/1a756cc77b5f4cbebd3a0b36cb19bf9a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:11.909600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5aaec4-9f69-48b1-bb83-7392c16517d8 None admin] RBAC: Authorizing `identity:get_project(project_id=1a756cc77b5f4cbebd3a0b36cb19bf9a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:11.911924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5aaec4-9f69-48b1-bb83-7392c16517d8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:11.914458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 587/1191] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:50:11 2022] GET /identity/v3/projects/1a756cc77b5f4cbebd3a0b36cb19bf9a => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:11.922583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036784c-5124-46b4-9524-baa5d7bf6c6e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:12.026599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3036784c-5124-46b4-9524-baa5d7bf6c6e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:12.062496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c3792-47de-4d4a-838f-7c648edbcf18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:12.063141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c3792-47de-4d4a-838f-7c648edbcf18 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:12.063418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c3792-47de-4d4a-838f-7c648edbcf18 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:12.063691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c3792-47de-4d4a-838f-7c648edbcf18 None admin] PATH_INFO: `/v3/projects/1a756cc77b5f4cbebd3a0b36cb19bf9a/users/4ac0e9c24af24b50a3d3731fa6cac7dd/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:12.065859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c3792-47de-4d4a-838f-7c648edbcf18 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a756cc77b5f4cbebd3a0b36cb19bf9a, user_id=4ac0e9c24af24b50a3d3731fa6cac7dd, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:12.067330 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c3792-47de-4d4a-838f-7c648edbcf18 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:12.087240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 605/1192] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:50:11 2022] PUT /identity/v3/projects/1a756cc77b5f4cbebd3a0b36cb19bf9a/users/4ac0e9c24af24b50a3d3731fa6cac7dd/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:50:14.215488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39fbf1-49ad-4ccc-8348-5003d3234f82 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:14.216611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39fbf1-49ad-4ccc-8348-5003d3234f82 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:14.216611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a39fbf1-49ad-4ccc-8348-5003d3234f82 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:14.218577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 588/1193] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:50:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:50:14.224866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529b193-fa41-49d6-a871-3a9191ce025a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:14.225308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529b193-fa41-49d6-a871-3a9191ce025a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:14.225424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a529b193-fa41-49d6-a871-3a9191ce025a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:14.247304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a529b193-fa41-49d6-a871-3a9191ce025a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:50:14.248505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a529b193-fa41-49d6-a871-3a9191ce025a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:50:14.278053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:14.329377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 606/1194] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:50:14 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:50:14.596949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39fbf1-49ad-4ccc-8348-5003d3234f82 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:14.712152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a39fbf1-49ad-4ccc-8348-5003d3234f82 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:14.747788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450642b-4ea1-45fd-8ae2-42fa9afabbb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:14.748406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450642b-4ea1-45fd-8ae2-42fa9afabbb9 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:14.748689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450642b-4ea1-45fd-8ae2-42fa9afabbb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:14.748958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4450642b-4ea1-45fd-8ae2-42fa9afabbb9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:14.759831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4450642b-4ea1-45fd-8ae2-42fa9afabbb9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-4450642b-4ea1-45fd-8ae2-42fa9afabbb9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:50:14.764758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:50:14.765826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:50:14.767107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 589/1195] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:50:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:50:14.775416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529b193-fa41-49d6-a871-3a9191ce025a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:14.887195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a529b193-fa41-49d6-a871-3a9191ce025a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:14.923116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649888a-1457-45aa-8773-f4cd1bd70e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:14.923720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649888a-1457-45aa-8773-f4cd1bd70e77 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:14.923995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649888a-1457-45aa-8773-f4cd1bd70e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:14.924235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2649888a-1457-45aa-8773-f4cd1bd70e77 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:14.924914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649888a-1457-45aa-8773-f4cd1bd70e77 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:50:14.925213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649888a-1457-45aa-8773-f4cd1bd70e77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:14.926629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2649888a-1457-45aa-8773-f4cd1bd70e77 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:14.936632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 607/1196] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:50:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:14.945986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450642b-4ea1-45fd-8ae2-42fa9afabbb9 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:14.999100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:15.054321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4450642b-4ea1-45fd-8ae2-42fa9afabbb9 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:15.090056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c354c8-47a5-4b05-9ec4-7e75ee6de944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:15.090635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c354c8-47a5-4b05-9ec4-7e75ee6de944 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:15.090949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c354c8-47a5-4b05-9ec4-7e75ee6de944 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:15.091220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c354c8-47a5-4b05-9ec4-7e75ee6de944 None admin] PATH_INFO: `/v3/users/4ac0e9c24af24b50a3d3731fa6cac7dd` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:15.092452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c354c8-47a5-4b05-9ec4-7e75ee6de944 None admin] RBAC: Authorizing `identity:get_user(user_id=4ac0e9c24af24b50a3d3731fa6cac7dd)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:15.093891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c354c8-47a5-4b05-9ec4-7e75ee6de944 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:15.095488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 590/1197] 10.222.0.25 () {56 vars in 1177 bytes} [Tue Sep 13 06:50:14 2022] GET /identity/v3/users/4ac0e9c24af24b50a3d3731fa6cac7dd => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:15.102302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649888a-1457-45aa-8773-f4cd1bd70e77 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:15.131193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:15.210035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2649888a-1457-45aa-8773-f4cd1bd70e77 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:15.247736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537b26f-b43d-4aff-a87c-a0e1be27ab84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:15.248353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537b26f-b43d-4aff-a87c-a0e1be27ab84 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:15.248597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537b26f-b43d-4aff-a87c-a0e1be27ab84 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:15.248868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e537b26f-b43d-4aff-a87c-a0e1be27ab84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:15.320456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537b26f-b43d-4aff-a87c-a0e1be27ab84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:15.321911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e537b26f-b43d-4aff-a87c-a0e1be27ab84 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:15.347221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:15.424015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 608/1198] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:50:15 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:50:15.430367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c354c8-47a5-4b05-9ec4-7e75ee6de944 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:15.533604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c354c8-47a5-4b05-9ec4-7e75ee6de944 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:15.553659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:15.575508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d255bb25-90bd-4965-8e6c-3b719506e24b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:15.576431 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255bb25-90bd-4965-8e6c-3b719506e24b None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:15.576795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255bb25-90bd-4965-8e6c-3b719506e24b None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:15.577149 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d255bb25-90bd-4965-8e6c-3b719506e24b None admin] PATH_INFO: `/v3/projects/1a756cc77b5f4cbebd3a0b36cb19bf9a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:15.578936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255bb25-90bd-4965-8e6c-3b719506e24b None admin] RBAC: Authorizing `identity:get_project(project_id=1a756cc77b5f4cbebd3a0b36cb19bf9a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:15.581114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d255bb25-90bd-4965-8e6c-3b719506e24b None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:15.583319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 591/1199] 10.222.0.25 () {56 vars in 1186 bytes} [Tue Sep 13 06:50:15 2022] GET /identity/v3/projects/1a756cc77b5f4cbebd3a0b36cb19bf9a => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:15.590942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537b26f-b43d-4aff-a87c-a0e1be27ab84 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:50:15.695080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e537b26f-b43d-4aff-a87c-a0e1be27ab84 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:50:15.718529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:15.732001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b875612-ad1c-4da1-b55b-74890608f148 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:50:15.732613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b875612-ad1c-4da1-b55b-74890608f148 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:50:15.732889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b875612-ad1c-4da1-b55b-74890608f148 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:50:15.733157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b875612-ad1c-4da1-b55b-74890608f148 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:50:15.733880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b875612-ad1c-4da1-b55b-74890608f148 None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=1a756cc77b5f4cbebd3a0b36cb19bf9a, user.id=4ac0e9c24af24b50a3d3731fa6cac7dd) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:50:15.734218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b875612-ad1c-4da1-b55b-74890608f148 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:50:15.735665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b875612-ad1c-4da1-b55b-74890608f148 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:50:15.741896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 609/1200] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:50:15 2022] GET /identity/v3/role_assignments?user.id=4ac0e9c24af24b50a3d3731fa6cac7dd&scope.project.id=1a756cc77b5f4cbebd3a0b36cb19bf9a&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:50:25.568777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=8 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:50:25.738973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:11.783933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e1d5-e8a7-4178-9d44-966926a3728d None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:11.784626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e1d5-e8a7-4178-9d44-966926a3728d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:11.784626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329e1d5-e8a7-4178-9d44-966926a3728d None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:11.786204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 592/1201] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:11.791819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5e05d-208e-4b1e-a8f4-0213e2db5fe5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:11.792304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5e05d-208e-4b1e-a8f4-0213e2db5fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:11.792393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f5e05d-208e-4b1e-a8f4-0213e2db5fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:11.813143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d9f5e05d-208e-4b1e-a8f4-0213e2db5fe5 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:11.815092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9f5e05d-208e-4b1e-a8f4-0213e2db5fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:11.921731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 610/1202] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:51:12.198036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329e1d5-e8a7-4178-9d44-966926a3728d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:12.320842 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329e1d5-e8a7-4178-9d44-966926a3728d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:12.362676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21406d2f-c795-4777-8dca-199609297064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:12.363347 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21406d2f-c795-4777-8dca-199609297064 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:12.363624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21406d2f-c795-4777-8dca-199609297064 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:12.363891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21406d2f-c795-4777-8dca-199609297064 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:12.376566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21406d2f-c795-4777-8dca-199609297064 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:12.378250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21406d2f-c795-4777-8dca-199609297064 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-21406d2f-c795-4777-8dca-199609297064 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:12.394582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:12.395742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:12.397424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:12.397424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:12.397424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:12.397424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 593/1203] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:51:12.402518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5e05d-208e-4b1e-a8f4-0213e2db5fe5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:12.515976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f5e05d-208e-4b1e-a8f4-0213e2db5fe5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:12.548359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:12.555582 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ba9055-2f69-4dbc-8da2-f33eca4811f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:12.556230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba9055-2f69-4dbc-8da2-f33eca4811f9 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:12.556534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba9055-2f69-4dbc-8da2-f33eca4811f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:12.556800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ba9055-2f69-4dbc-8da2-f33eca4811f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:12.557582 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba9055-2f69-4dbc-8da2-f33eca4811f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:12.558287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba9055-2f69-4dbc-8da2-f33eca4811f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:12.560204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ba9055-2f69-4dbc-8da2-f33eca4811f9 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:12.578289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 611/1204] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:12.579700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21406d2f-c795-4777-8dca-199609297064 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:12.661851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:12.729266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21406d2f-c795-4777-8dca-199609297064 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:12.786387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed62e70-c24a-4599-8806-dcbf931dd0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:12.787074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed62e70-c24a-4599-8806-dcbf931dd0ae None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:12.787074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed62e70-c24a-4599-8806-dcbf931dd0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:12.787353 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed62e70-c24a-4599-8806-dcbf931dd0ae None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:12.788027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed62e70-c24a-4599-8806-dcbf931dd0ae None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:12.789471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed62e70-c24a-4599-8806-dcbf931dd0ae None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:12.820677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 594/1205] 10.222.0.25 () {60 vars in 1132 bytes} [Tue Sep 13 06:51:12 2022] POST /identity/v3/users => generated 251 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:51:15.243757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62f617-e492-4eac-9b7a-41f23435ec1f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:15.243757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62f617-e492-4eac-9b7a-41f23435ec1f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:15.243757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a62f617-e492-4eac-9b7a-41f23435ec1f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:15.243757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 612/1206] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:51:15.249390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59518584-f3b3-4264-b7d6-b8a1815957a4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:15.249994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59518584-f3b3-4264-b7d6-b8a1815957a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:15.249994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59518584-f3b3-4264-b7d6-b8a1815957a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:15.269146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-59518584-f3b3-4264-b7d6-b8a1815957a4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:15.271275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-59518584-f3b3-4264-b7d6-b8a1815957a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:15.348629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 595/1207] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:51:15.612604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62f617-e492-4eac-9b7a-41f23435ec1f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:15.737430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a62f617-e492-4eac-9b7a-41f23435ec1f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:15.773179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4aa8b-75df-4e37-a4b4-37f025e65003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:15.773920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4aa8b-75df-4e37-a4b4-37f025e65003 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:15.774212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4aa8b-75df-4e37-a4b4-37f025e65003 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:15.774529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd4aa8b-75df-4e37-a4b4-37f025e65003 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:15.786931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd4aa8b-75df-4e37-a4b4-37f025e65003 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-3fd4aa8b-75df-4e37-a4b4-37f025e65003 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:51:15.791840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:15.792925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:15.793915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 613/1208] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:51:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:15.804529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59518584-f3b3-4264-b7d6-b8a1815957a4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:15.864684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:15.944530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59518584-f3b3-4264-b7d6-b8a1815957a4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:15.982888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26f2b-e8bd-4e5a-814b-7f9181e478b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:15.983513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26f2b-e8bd-4e5a-814b-7f9181e478b5 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:15.983793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26f2b-e8bd-4e5a-814b-7f9181e478b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:15.984067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26f2b-e8bd-4e5a-814b-7f9181e478b5 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:15.984742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26f2b-e8bd-4e5a-814b-7f9181e478b5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:15.985061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26f2b-e8bd-4e5a-814b-7f9181e478b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:15.986527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26f2b-e8bd-4e5a-814b-7f9181e478b5 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:15.996431 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 596/1209] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:51:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:16.002889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4aa8b-75df-4e37-a4b4-37f025e65003 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:16.062430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:16.129755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd4aa8b-75df-4e37-a4b4-37f025e65003 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:16.175465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e946e-9af3-4057-8a7a-6b6e79b32493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:16.176149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e946e-9af3-4057-8a7a-6b6e79b32493 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:16.176848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e946e-9af3-4057-8a7a-6b6e79b32493 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:16.176848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e946e-9af3-4057-8a7a-6b6e79b32493 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:16.188675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e946e-9af3-4057-8a7a-6b6e79b32493 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:16.190181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e946e-9af3-4057-8a7a-6b6e79b32493 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-8a6e946e-9af3-4057-8a7a-6b6e79b32493 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:16.205991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:16.207337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:16.208395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:16.208395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:16.208395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:16.212011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 614/1210] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:16.213521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26f2b-e8bd-4e5a-814b-7f9181e478b5 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:16.348082 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:16.351817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26f2b-e8bd-4e5a-814b-7f9181e478b5 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:16.399173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad944f8-7bbb-4c94-ad9f-d5e7b79735c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:16.399173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad944f8-7bbb-4c94-ad9f-d5e7b79735c1 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:16.399173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad944f8-7bbb-4c94-ad9f-d5e7b79735c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:16.399173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad944f8-7bbb-4c94-ad9f-d5e7b79735c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:16.399632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad944f8-7bbb-4c94-ad9f-d5e7b79735c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:16.399632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad944f8-7bbb-4c94-ad9f-d5e7b79735c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:16.399790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad944f8-7bbb-4c94-ad9f-d5e7b79735c1 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:16.414294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 597/1211] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:16.422135 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e946e-9af3-4057-8a7a-6b6e79b32493 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:16.537172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e946e-9af3-4057-8a7a-6b6e79b32493 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:16.539070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:16.574591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384d83e-98f3-4ed4-8cb2-cfaaa8efe47e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:16.575426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384d83e-98f3-4ed4-8cb2-cfaaa8efe47e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:16.575777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384d83e-98f3-4ed4-8cb2-cfaaa8efe47e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:16.576124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d384d83e-98f3-4ed4-8cb2-cfaaa8efe47e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:16.584667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384d83e-98f3-4ed4-8cb2-cfaaa8efe47e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:16.586294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d384d83e-98f3-4ed4-8cb2-cfaaa8efe47e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d384d83e-98f3-4ed4-8cb2-cfaaa8efe47e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:51:16.602386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:16.603915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:16.605123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:16.610439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 615/1212] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:16.611982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad944f8-7bbb-4c94-ad9f-d5e7b79735c1 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:16.719207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad944f8-7bbb-4c94-ad9f-d5e7b79735c1 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:16.756635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d0f51-5673-4908-b3ec-0eb26e46e650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:16.757531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d0f51-5673-4908-b3ec-0eb26e46e650 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:16.757892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d0f51-5673-4908-b3ec-0eb26e46e650 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:16.758202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172d0f51-5673-4908-b3ec-0eb26e46e650 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:16.758977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d0f51-5673-4908-b3ec-0eb26e46e650 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:16.759331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d0f51-5673-4908-b3ec-0eb26e46e650 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:16.761965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172d0f51-5673-4908-b3ec-0eb26e46e650 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:16.780031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 598/1213] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Sep 13 06:51:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:16.786228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384d83e-98f3-4ed4-8cb2-cfaaa8efe47e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:16.891293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d384d83e-98f3-4ed4-8cb2-cfaaa8efe47e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:16.928790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec0071-f306-408c-9f4d-8706cb7a21f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:16.929405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec0071-f306-408c-9f4d-8706cb7a21f7 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:16.929682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec0071-f306-408c-9f4d-8706cb7a21f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:16.929950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aec0071-f306-408c-9f4d-8706cb7a21f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:16.963067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:17.010691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec0071-f306-408c-9f4d-8706cb7a21f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:17.012098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aec0071-f306-408c-9f4d-8706cb7a21f7 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:17.129910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 616/1214] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:51:16 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:51:17.136006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d0f51-5673-4908-b3ec-0eb26e46e650 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:17.158376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:17.258927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172d0f51-5673-4908-b3ec-0eb26e46e650 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:17.304852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4be5cb-5b50-46f5-99d7-498cf0d6508c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:17.305681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4be5cb-5b50-46f5-99d7-498cf0d6508c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:17.305879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4be5cb-5b50-46f5-99d7-498cf0d6508c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:17.306217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4be5cb-5b50-46f5-99d7-498cf0d6508c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:17.319038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4be5cb-5b50-46f5-99d7-498cf0d6508c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:17.320950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4be5cb-5b50-46f5-99d7-498cf0d6508c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1c4be5cb-5b50-46f5-99d7-498cf0d6508c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:17.338877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:17.340132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:17.341301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:17.341301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:17.341301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:17.341301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 599/1215] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:51:17.347817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec0071-f306-408c-9f4d-8706cb7a21f7 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:17.419828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:17.475221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aec0071-f306-408c-9f4d-8706cb7a21f7 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:17.516152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e79c4f-dd70-49e3-94d2-9ee63e682889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:17.516866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e79c4f-dd70-49e3-94d2-9ee63e682889 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:17.517217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e79c4f-dd70-49e3-94d2-9ee63e682889 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:17.517533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e79c4f-dd70-49e3-94d2-9ee63e682889 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:17.518424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e79c4f-dd70-49e3-94d2-9ee63e682889 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:17.518788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e79c4f-dd70-49e3-94d2-9ee63e682889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:17.520376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e79c4f-dd70-49e3-94d2-9ee63e682889 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:17.532240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 617/1216] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:17.537946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4be5cb-5b50-46f5-99d7-498cf0d6508c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:17.614598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:17.648436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4be5cb-5b50-46f5-99d7-498cf0d6508c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:17.685239 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49bb07-bc4a-4b88-a926-7bd1af452ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:17.685896 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49bb07-bc4a-4b88-a926-7bd1af452ced None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:17.686177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49bb07-bc4a-4b88-a926-7bd1af452ced None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:17.686446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac49bb07-bc4a-4b88-a926-7bd1af452ced None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:17.693358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49bb07-bc4a-4b88-a926-7bd1af452ced None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:17.694844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac49bb07-bc4a-4b88-a926-7bd1af452ced None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-ac49bb07-bc4a-4b88-a926-7bd1af452ced None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:51:17.706479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:17.707595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:17.708664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:51:17.708664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:51:17.708664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:51:17.708664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:51:17.708664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:51:17.708664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:51:17.708664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:17.708664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:17.708664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 600/1217] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:51:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:17.713747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e79c4f-dd70-49e3-94d2-9ee63e682889 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:17.818822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e79c4f-dd70-49e3-94d2-9ee63e682889 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:17.845631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:17.859555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4f2c36-b91e-4467-b688-17b48e64732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:17.860171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f2c36-b91e-4467-b688-17b48e64732f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:17.860497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f2c36-b91e-4467-b688-17b48e64732f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:17.860713 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4f2c36-b91e-4467-b688-17b48e64732f None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:17.861476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f2c36-b91e-4467-b688-17b48e64732f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:17.861704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f2c36-b91e-4467-b688-17b48e64732f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:17.863191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4f2c36-b91e-4467-b688-17b48e64732f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:17.876518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 618/1218] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:51:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:17.883003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49bb07-bc4a-4b88-a926-7bd1af452ced None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:17.997916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac49bb07-bc4a-4b88-a926-7bd1af452ced None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:18.033344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:18.039767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52101089-e6ee-416a-9bb9-745ef594b9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:18.043124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52101089-e6ee-416a-9bb9-745ef594b9ee None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:18.043124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52101089-e6ee-416a-9bb9-745ef594b9ee None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:18.043124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52101089-e6ee-416a-9bb9-745ef594b9ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:18.043124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52101089-e6ee-416a-9bb9-745ef594b9ee None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=e8202ad6007246078c4d25365dc5a0f5) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:18.043124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52101089-e6ee-416a-9bb9-745ef594b9ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:18.043738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52101089-e6ee-416a-9bb9-745ef594b9ee None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:18.049097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 601/1219] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:51:17 2022] GET /identity/v3/role_assignments?user.id=e8202ad6007246078c4d25365dc5a0f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:20.353453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02ee67-e237-4062-8aa6-70a631062b49 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:20.354111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02ee67-e237-4062-8aa6-70a631062b49 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:20.354329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac02ee67-e237-4062-8aa6-70a631062b49 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:20.356538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 619/1220] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:20.363296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0c463-36fb-4229-a3cd-b19ec46539e1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:20.363756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0c463-36fb-4229-a3cd-b19ec46539e1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:20.363865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed0c463-36fb-4229-a3cd-b19ec46539e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:20.387910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4ed0c463-36fb-4229-a3cd-b19ec46539e1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:20.390753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4ed0c463-36fb-4229-a3cd-b19ec46539e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:20.464375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 602/1221] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:51:20.734468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02ee67-e237-4062-8aa6-70a631062b49 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:20.840502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac02ee67-e237-4062-8aa6-70a631062b49 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:20.876051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241642fe-f11c-40f7-8aef-50b10466fc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:20.876957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241642fe-f11c-40f7-8aef-50b10466fc10 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:20.877234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241642fe-f11c-40f7-8aef-50b10466fc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:20.877508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241642fe-f11c-40f7-8aef-50b10466fc10 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:20.888577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-241642fe-f11c-40f7-8aef-50b10466fc10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-241642fe-f11c-40f7-8aef-50b10466fc10 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:51:20.893469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:20.894772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:51:20.895898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:20.899816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 620/1222] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:51:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:51:20.901236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0c463-36fb-4229-a3cd-b19ec46539e1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:21.012656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed0c463-36fb-4229-a3cd-b19ec46539e1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:21.038828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:21.051094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6881054-8ad4-4e79-a4e8-a6452692c243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:21.051715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6881054-8ad4-4e79-a4e8-a6452692c243 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:21.051998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6881054-8ad4-4e79-a4e8-a6452692c243 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:21.052269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6881054-8ad4-4e79-a4e8-a6452692c243 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:21.052973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6881054-8ad4-4e79-a4e8-a6452692c243 None admin] RBAC: Adding query filter params (name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:21.053300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6881054-8ad4-4e79-a4e8-a6452692c243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:21.054707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6881054-8ad4-4e79-a4e8-a6452692c243 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:21.063963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 603/1223] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:51:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:21.070156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241642fe-f11c-40f7-8aef-50b10466fc10 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:21.122227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:21.185047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-241642fe-f11c-40f7-8aef-50b10466fc10 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:21.225411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066f427-936e-4c34-bebe-6fa994ea887a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:21.226195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066f427-936e-4c34-bebe-6fa994ea887a None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:21.226481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066f427-936e-4c34-bebe-6fa994ea887a None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:21.226756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2066f427-936e-4c34-bebe-6fa994ea887a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:21.237483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066f427-936e-4c34-bebe-6fa994ea887a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:21.239052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2066f427-936e-4c34-bebe-6fa994ea887a None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-2066f427-936e-4c34-bebe-6fa994ea887a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:21.255282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:21.256861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:21.258337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:21.258337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:21.258337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:21.262951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6881054-8ad4-4e79-a4e8-a6452692c243 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:21.263410 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 621/1224] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:21.377216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6881054-8ad4-4e79-a4e8-a6452692c243 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:21.412925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af8ea0-b3dd-42ff-a181-e50b6ea3786c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:21.413543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8ea0-b3dd-42ff-a181-e50b6ea3786c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:21.413819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8ea0-b3dd-42ff-a181-e50b6ea3786c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:21.414106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84af8ea0-b3dd-42ff-a181-e50b6ea3786c None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:21.414747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af8ea0-b3dd-42ff-a181-e50b6ea3786c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:21.415050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af8ea0-b3dd-42ff-a181-e50b6ea3786c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:21.416419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84af8ea0-b3dd-42ff-a181-e50b6ea3786c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:21.428741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 604/1225] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:51:21.435811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066f427-936e-4c34-bebe-6fa994ea887a None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:21.548451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:21.556060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2066f427-936e-4c34-bebe-6fa994ea887a None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:21.594454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feecb8c-7730-41bd-8964-56ba2128987e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:21.595118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feecb8c-7730-41bd-8964-56ba2128987e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:21.595367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feecb8c-7730-41bd-8964-56ba2128987e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:21.595667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6feecb8c-7730-41bd-8964-56ba2128987e None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:21.604739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feecb8c-7730-41bd-8964-56ba2128987e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:21.606635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6feecb8c-7730-41bd-8964-56ba2128987e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-6feecb8c-7730-41bd-8964-56ba2128987e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:51:21.618933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:21.620140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:21.621384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 622/1226] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:51:21.626402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af8ea0-b3dd-42ff-a181-e50b6ea3786c None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:21.675515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:21.737461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84af8ea0-b3dd-42ff-a181-e50b6ea3786c None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:21.772011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0007f-0397-4445-b2a9-841d66f64054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:21.772620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0007f-0397-4445-b2a9-841d66f64054 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:21.772890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0007f-0397-4445-b2a9-841d66f64054 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:21.773155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a0007f-0397-4445-b2a9-841d66f64054 None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:21.773878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0007f-0397-4445-b2a9-841d66f64054 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:21.774211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0007f-0397-4445-b2a9-841d66f64054 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:21.775642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a0007f-0397-4445-b2a9-841d66f64054 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:21.795836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 605/1227] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Sep 13 06:51:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:21.801948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feecb8c-7730-41bd-8964-56ba2128987e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:21.908133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6feecb8c-7730-41bd-8964-56ba2128987e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:21.942883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849be1c-a65b-46d0-b9b5-b99f3032af07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:21.943521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849be1c-a65b-46d0-b9b5-b99f3032af07 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:21.943799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849be1c-a65b-46d0-b9b5-b99f3032af07 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:21.944072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4849be1c-a65b-46d0-b9b5-b99f3032af07 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:21.955532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849be1c-a65b-46d0-b9b5-b99f3032af07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:21.956987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4849be1c-a65b-46d0-b9b5-b99f3032af07 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4849be1c-a65b-46d0-b9b5-b99f3032af07 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:21.971495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:21.972693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:21.973773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:21.973773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:21.973773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:21.973773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 623/1228] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:21.978595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0007f-0397-4445-b2a9-841d66f64054 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:22.097774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a0007f-0397-4445-b2a9-841d66f64054 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:22.101492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:22.141596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8f46f-c28e-4672-a3b6-41502779f53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:22.142638 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8f46f-c28e-4672-a3b6-41502779f53a None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:22.142911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8f46f-c28e-4672-a3b6-41502779f53a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:22.143268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f8f46f-c28e-4672-a3b6-41502779f53a None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:22.143916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8f46f-c28e-4672-a3b6-41502779f53a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:22.144318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8f46f-c28e-4672-a3b6-41502779f53a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:22.145833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f8f46f-c28e-4672-a3b6-41502779f53a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:22.159746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 606/1229] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:22.165615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849be1c-a65b-46d0-b9b5-b99f3032af07 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:22.176422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:22.276591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4849be1c-a65b-46d0-b9b5-b99f3032af07 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:22.312399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0181ea-af58-4415-b8f1-574d90030400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:22.313038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0181ea-af58-4415-b8f1-574d90030400 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:22.313318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0181ea-af58-4415-b8f1-574d90030400 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:22.313661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0181ea-af58-4415-b8f1-574d90030400 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:22.321382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0181ea-af58-4415-b8f1-574d90030400 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:22.322862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0181ea-af58-4415-b8f1-574d90030400 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ca0181ea-af58-4415-b8f1-574d90030400 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:51:22.336378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:22.337541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:22.338844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:51:22.338844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:51:22.338844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:51:22.338844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:51:22.338844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:51:22.338844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:51:22.338844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:22.338844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:22.338844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 624/1230] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:51:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:22.344421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8f46f-c28e-4672-a3b6-41502779f53a None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:22.456368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f8f46f-c28e-4672-a3b6-41502779f53a None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:22.496001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4196f5ef-2911-4918-853b-d586e1d216b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:22.497093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196f5ef-2911-4918-853b-d586e1d216b7 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:22.497500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196f5ef-2911-4918-853b-d586e1d216b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:22.497837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4196f5ef-2911-4918-853b-d586e1d216b7 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:22.498553 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196f5ef-2911-4918-853b-d586e1d216b7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:22.498858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196f5ef-2911-4918-853b-d586e1d216b7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:22.500300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4196f5ef-2911-4918-853b-d586e1d216b7 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:22.513169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 607/1231] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:51:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:22.519631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0181ea-af58-4415-b8f1-574d90030400 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:22.589612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:22.638905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0181ea-af58-4415-b8f1-574d90030400 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:22.681115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5524068-8d2c-49f1-91c2-1b73ba0f94b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:22.681936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5524068-8d2c-49f1-91c2-1b73ba0f94b8 None admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:22.682264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5524068-8d2c-49f1-91c2-1b73ba0f94b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:22.682525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5524068-8d2c-49f1-91c2-1b73ba0f94b8 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/e8202ad6007246078c4d25365dc5a0f5/roles/df32bbf3791f4028b2dabf497d5e938d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:22.698323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5524068-8d2c-49f1-91c2-1b73ba0f94b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=e8202ad6007246078c4d25365dc5a0f5, role_id=df32bbf3791f4028b2dabf497d5e938d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:22.699847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5524068-8d2c-49f1-91c2-1b73ba0f94b8 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:22.713206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 625/1232] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:51:22 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/e8202ad6007246078c4d25365dc5a0f5/roles/df32bbf3791f4028b2dabf497d5e938d => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:51:24.843643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc18967-2693-4580-9bc2-c17ffad9620c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:24.844261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc18967-2693-4580-9bc2-c17ffad9620c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:24.844261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc18967-2693-4580-9bc2-c17ffad9620c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:24.845569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 608/1233] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:24.851359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc399b3-1947-47b8-b3ad-206cf38b5876 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:24.851830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc399b3-1947-47b8-b3ad-206cf38b5876 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:24.851925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc399b3-1947-47b8-b3ad-206cf38b5876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:24.870777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fcc399b3-1947-47b8-b3ad-206cf38b5876 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:24.872009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fcc399b3-1947-47b8-b3ad-206cf38b5876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:24.943490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 626/1234] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:51:25.212498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc18967-2693-4580-9bc2-c17ffad9620c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:25.242337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:25.336115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc18967-2693-4580-9bc2-c17ffad9620c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:25.373420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f37e3-bab1-4fbf-9deb-15969d012c74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:25.374067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f37e3-bab1-4fbf-9deb-15969d012c74 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:25.374375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f37e3-bab1-4fbf-9deb-15969d012c74 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:25.374647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390f37e3-bab1-4fbf-9deb-15969d012c74 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:25.385109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390f37e3-bab1-4fbf-9deb-15969d012c74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-390f37e3-bab1-4fbf-9deb-15969d012c74 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:51:25.390769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:25.392192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:25.393283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 609/1235] 10.222.0.25 () {56 vars in 1102 bytes} [Tue Sep 13 06:51:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:25.398111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc399b3-1947-47b8-b3ad-206cf38b5876 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:25.446622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:25.507374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc399b3-1947-47b8-b3ad-206cf38b5876 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:25.542423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abdd86-e23a-481a-9ce1-6321b23fc31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:25.543043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abdd86-e23a-481a-9ce1-6321b23fc31e None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:25.543418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abdd86-e23a-481a-9ce1-6321b23fc31e None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:25.543696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22abdd86-e23a-481a-9ce1-6321b23fc31e None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:25.544364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abdd86-e23a-481a-9ce1-6321b23fc31e None admin] RBAC: Adding query filter params (name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:25.544668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abdd86-e23a-481a-9ce1-6321b23fc31e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:25.546132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22abdd86-e23a-481a-9ce1-6321b23fc31e None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:25.554150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 627/1236] 10.222.0.25 () {56 vars in 1103 bytes} [Tue Sep 13 06:51:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:25.560566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f37e3-bab1-4fbf-9deb-15969d012c74 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:25.655827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:25.674308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390f37e3-bab1-4fbf-9deb-15969d012c74 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:25.710881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ed336-c0a2-4ca1-ade3-34c83312ca29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:25.711587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ed336-c0a2-4ca1-ade3-34c83312ca29 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:25.711925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ed336-c0a2-4ca1-ade3-34c83312ca29 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:25.712194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ed336-c0a2-4ca1-ade3-34c83312ca29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:25.723342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ed336-c0a2-4ca1-ade3-34c83312ca29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:25.724801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ed336-c0a2-4ca1-ade3-34c83312ca29 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-831ed336-c0a2-4ca1-ade3-34c83312ca29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:25.740066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:25.741365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:25.742383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:25.742383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:25.742383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:25.742383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 610/1237] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:25.748386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abdd86-e23a-481a-9ce1-6321b23fc31e None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:25.860897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22abdd86-e23a-481a-9ce1-6321b23fc31e None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:25.870386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:25.895682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe88402-ee7f-4594-978e-3867e48434c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:25.896294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe88402-ee7f-4594-978e-3867e48434c4 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:25.896594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe88402-ee7f-4594-978e-3867e48434c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:25.896880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe88402-ee7f-4594-978e-3867e48434c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:25.897555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe88402-ee7f-4594-978e-3867e48434c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:25.897819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe88402-ee7f-4594-978e-3867e48434c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:25.899273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe88402-ee7f-4594-978e-3867e48434c4 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:25.911721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 628/1238] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:25.917552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ed336-c0a2-4ca1-ade3-34c83312ca29 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:26.022098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ed336-c0a2-4ca1-ade3-34c83312ca29 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:26.060519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322de79a-faa2-442f-8330-2483882457f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:26.061166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322de79a-faa2-442f-8330-2483882457f8 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:26.061443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322de79a-faa2-442f-8330-2483882457f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:26.061717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322de79a-faa2-442f-8330-2483882457f8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:26.064531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:26.070222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322de79a-faa2-442f-8330-2483882457f8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:26.071870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322de79a-faa2-442f-8330-2483882457f8 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-322de79a-faa2-442f-8330-2483882457f8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:51:26.083461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:26.084598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:26.085708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 611/1239] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:26.090918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe88402-ee7f-4594-978e-3867e48434c4 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:26.201753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe88402-ee7f-4594-978e-3867e48434c4 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:26.241720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2ab83-8419-4afd-863e-4ff58308f17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:26.242409 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2ab83-8419-4afd-863e-4ff58308f17d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:26.242682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2ab83-8419-4afd-863e-4ff58308f17d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:26.242938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2ab83-8419-4afd-863e-4ff58308f17d None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:26.243672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2ab83-8419-4afd-863e-4ff58308f17d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:26.244021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2ab83-8419-4afd-863e-4ff58308f17d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:26.245413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b2ab83-8419-4afd-863e-4ff58308f17d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:26.267864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 629/1240] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Sep 13 06:51:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:26.274217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322de79a-faa2-442f-8330-2483882457f8 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:26.389711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322de79a-faa2-442f-8330-2483882457f8 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:26.427828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a594a-c1e6-4f4a-84ac-c64ac48908ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:26.428533 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a594a-c1e6-4f4a-84ac-c64ac48908ee None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:26.428848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a594a-c1e6-4f4a-84ac-c64ac48908ee None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:26.429156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712a594a-c1e6-4f4a-84ac-c64ac48908ee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:26.507401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:26.514379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a594a-c1e6-4f4a-84ac-c64ac48908ee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:26.515849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712a594a-c1e6-4f4a-84ac-c64ac48908ee None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:26.619880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 612/1241] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:51:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:51:26.626413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2ab83-8419-4afd-863e-4ff58308f17d None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:26.654408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:26.748291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b2ab83-8419-4afd-863e-4ff58308f17d None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:26.805341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc68ae4-dc4e-43d7-bcc7-e70379fd142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:26.806345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc68ae4-dc4e-43d7-bcc7-e70379fd142c None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:26.806818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc68ae4-dc4e-43d7-bcc7-e70379fd142c None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:26.807282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc68ae4-dc4e-43d7-bcc7-e70379fd142c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:26.826139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc68ae4-dc4e-43d7-bcc7-e70379fd142c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:26.827626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bc68ae4-dc4e-43d7-bcc7-e70379fd142c None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1bc68ae4-dc4e-43d7-bcc7-e70379fd142c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:26.846324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:26.847581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:26.848677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:26.848677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:26.848677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:26.848677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 630/1242] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:51:26.853463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a594a-c1e6-4f4a-84ac-c64ac48908ee None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:26.959855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:26.966225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712a594a-c1e6-4f4a-84ac-c64ac48908ee None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:27.004187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ee755-1a8c-4f3c-a5f3-69f0e057546d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:27.004802 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ee755-1a8c-4f3c-a5f3-69f0e057546d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:27.005084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ee755-1a8c-4f3c-a5f3-69f0e057546d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:27.005359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ee755-1a8c-4f3c-a5f3-69f0e057546d None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:27.005999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ee755-1a8c-4f3c-a5f3-69f0e057546d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:27.006326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ee755-1a8c-4f3c-a5f3-69f0e057546d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:27.007699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ee755-1a8c-4f3c-a5f3-69f0e057546d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:27.020640 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 613/1243] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:27.026744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc68ae4-dc4e-43d7-bcc7-e70379fd142c None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:27.113772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:27.141014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc68ae4-dc4e-43d7-bcc7-e70379fd142c None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:27.181213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1625b2-7639-4c21-8a93-e5eec50581ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:27.181879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1625b2-7639-4c21-8a93-e5eec50581ba None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:27.182246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1625b2-7639-4c21-8a93-e5eec50581ba None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:27.182519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1625b2-7639-4c21-8a93-e5eec50581ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:27.190796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1625b2-7639-4c21-8a93-e5eec50581ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:27.192278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1625b2-7639-4c21-8a93-e5eec50581ba None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-9e1625b2-7639-4c21-8a93-e5eec50581ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:51:27.205930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:27.207213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:27.208147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:51:27.208147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:51:27.208147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:51:27.208147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:51:27.208147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:51:27.208147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:51:27.208147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:27.208147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:27.208147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 631/1244] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:51:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:27.213379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ee755-1a8c-4f3c-a5f3-69f0e057546d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:27.328399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ee755-1a8c-4f3c-a5f3-69f0e057546d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:27.367153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c1d850-97f6-4f27-9159-34094ee69f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:27.367730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1d850-97f6-4f27-9159-34094ee69f7c None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:27.368090 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1d850-97f6-4f27-9159-34094ee69f7c None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:27.368366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c1d850-97f6-4f27-9159-34094ee69f7c None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:27.369168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1d850-97f6-4f27-9159-34094ee69f7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:27.369451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1d850-97f6-4f27-9159-34094ee69f7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:27.371280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c1d850-97f6-4f27-9159-34094ee69f7c None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:27.384403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 614/1245] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:51:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:27.391127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1625b2-7639-4c21-8a93-e5eec50581ba None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:27.502680 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1625b2-7639-4c21-8a93-e5eec50581ba None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:27.539978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b278f684-6212-4a01-8ce7-369c14103027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:27.542247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278f684-6212-4a01-8ce7-369c14103027 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:27.542531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278f684-6212-4a01-8ce7-369c14103027 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:27.542804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b278f684-6212-4a01-8ce7-369c14103027 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:27.543835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278f684-6212-4a01-8ce7-369c14103027 None admin] RBAC: Adding query filter params (role.id=df32bbf3791f4028b2dabf497d5e938d, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=e8202ad6007246078c4d25365dc5a0f5) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:27.544151 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278f684-6212-4a01-8ce7-369c14103027 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:27.545609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b278f684-6212-4a01-8ce7-369c14103027 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:27.552091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 632/1246] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:51:27 2022] GET /identity/v3/role_assignments?user.id=e8202ad6007246078c4d25365dc5a0f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=df32bbf3791f4028b2dabf497d5e938d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:30.022434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc146eec-3cf2-401a-9620-aab88bf09820 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:30.022979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc146eec-3cf2-401a-9620-aab88bf09820 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:30.022979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc146eec-3cf2-401a-9620-aab88bf09820 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:30.024499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 615/1247] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:30.030760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73c03b-722a-40d5-848f-a83441174dba None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:30.031443 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73c03b-722a-40d5-848f-a83441174dba None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:30.031443 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73c03b-722a-40d5-848f-a83441174dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:30.037642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:30.051508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2a73c03b-722a-40d5-848f-a83441174dba None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:30.052765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2a73c03b-722a-40d5-848f-a83441174dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:30.131195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 633/1248] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:51:30.395215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc146eec-3cf2-401a-9620-aab88bf09820 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:30.420778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:30.523330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc146eec-3cf2-401a-9620-aab88bf09820 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:30.563633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4a3e2-8b89-4e8f-97c1-e1749bc1b683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:30.564405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4a3e2-8b89-4e8f-97c1-e1749bc1b683 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:30.564717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4a3e2-8b89-4e8f-97c1-e1749bc1b683 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:30.565016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd4a3e2-8b89-4e8f-97c1-e1749bc1b683 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:30.576598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd4a3e2-8b89-4e8f-97c1-e1749bc1b683 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1fd4a3e2-8b89-4e8f-97c1-e1749bc1b683 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:51:30.582221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:30.583227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:30.584093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 616/1249] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:51:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:51:30.590509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73c03b-722a-40d5-848f-a83441174dba None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:30.701404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73c03b-722a-40d5-848f-a83441174dba None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:30.735422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:30.741350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c2c30-7b66-4f6d-a49f-e5a4d7a86c12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:30.742370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c2c30-7b66-4f6d-a49f-e5a4d7a86c12 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:30.743032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c2c30-7b66-4f6d-a49f-e5a4d7a86c12 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:30.743630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c2c30-7b66-4f6d-a49f-e5a4d7a86c12 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:30.744367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c2c30-7b66-4f6d-a49f-e5a4d7a86c12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:30.744680 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c2c30-7b66-4f6d-a49f-e5a4d7a86c12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:30.746350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c2c30-7b66-4f6d-a49f-e5a4d7a86c12 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:30.756233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 634/1250] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:51:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:30.762777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4a3e2-8b89-4e8f-97c1-e1749bc1b683 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:30.848883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:30.876108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd4a3e2-8b89-4e8f-97c1-e1749bc1b683 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:30.917293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4023859-55eb-4b6b-949a-94e559af8f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:30.917293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4023859-55eb-4b6b-949a-94e559af8f43 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:30.917293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4023859-55eb-4b6b-949a-94e559af8f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:30.917704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4023859-55eb-4b6b-949a-94e559af8f43 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:30.929416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4023859-55eb-4b6b-949a-94e559af8f43 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:30.930998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4023859-55eb-4b6b-949a-94e559af8f43 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-c4023859-55eb-4b6b-949a-94e559af8f43 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:30.948291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:30.949706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:30.950781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:30.950781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:30.950781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:30.950781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 617/1251] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:30.955452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c2c30-7b66-4f6d-a49f-e5a4d7a86c12 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:31.064961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c2c30-7b66-4f6d-a49f-e5a4d7a86c12 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:31.103616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc22814-e5f3-45a9-9951-e2d9dc958768 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:31.104516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22814-e5f3-45a9-9951-e2d9dc958768 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:31.104958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22814-e5f3-45a9-9951-e2d9dc958768 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:31.105381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc22814-e5f3-45a9-9951-e2d9dc958768 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:31.106273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc22814-e5f3-45a9-9951-e2d9dc958768 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:31.106753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc22814-e5f3-45a9-9951-e2d9dc958768 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:31.108530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc22814-e5f3-45a9-9951-e2d9dc958768 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:31.122868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 635/1252] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:31.129109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4023859-55eb-4b6b-949a-94e559af8f43 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:31.244589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4023859-55eb-4b6b-949a-94e559af8f43 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:31.278412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:31.287528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184b2d2-54a4-4316-9dd7-971b3b3c0cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:31.288171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184b2d2-54a4-4316-9dd7-971b3b3c0cad None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:31.288474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184b2d2-54a4-4316-9dd7-971b3b3c0cad None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:31.288719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184b2d2-54a4-4316-9dd7-971b3b3c0cad None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:31.298383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184b2d2-54a4-4316-9dd7-971b3b3c0cad None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:31.300318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184b2d2-54a4-4316-9dd7-971b3b3c0cad None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-6184b2d2-54a4-4316-9dd7-971b3b3c0cad None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:51:31.314426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:31.315498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:31.316478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 618/1253] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:31.322177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc22814-e5f3-45a9-9951-e2d9dc958768 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:31.384192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:31.429585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc22814-e5f3-45a9-9951-e2d9dc958768 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:31.468100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c732c-5176-44fb-a533-26f326794641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:31.468750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c732c-5176-44fb-a533-26f326794641 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:31.469033 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c732c-5176-44fb-a533-26f326794641 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:31.469346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c732c-5176-44fb-a533-26f326794641 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:31.470059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c732c-5176-44fb-a533-26f326794641 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:31.470444 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c732c-5176-44fb-a533-26f326794641 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:31.471914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c732c-5176-44fb-a533-26f326794641 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:31.490912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 636/1254] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Sep 13 06:51:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:31.497205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184b2d2-54a4-4316-9dd7-971b3b3c0cad None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:31.610705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184b2d2-54a4-4316-9dd7-971b3b3c0cad None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:31.650900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef4339-03d9-4174-800c-31666039d34d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:31.651982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef4339-03d9-4174-800c-31666039d34d None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:31.652264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef4339-03d9-4174-800c-31666039d34d None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:31.652685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ef4339-03d9-4174-800c-31666039d34d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:31.717294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:31.729638 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef4339-03d9-4174-800c-31666039d34d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:31.731256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ef4339-03d9-4174-800c-31666039d34d None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:31.838890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 619/1255] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:51:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:51:31.844777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c732c-5176-44fb-a533-26f326794641 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:31.952751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c732c-5176-44fb-a533-26f326794641 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:31.967955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:31.990610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ce757-72bb-4bac-84af-05c6743ae168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:31.991252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ce757-72bb-4bac-84af-05c6743ae168 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:31.991530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ce757-72bb-4bac-84af-05c6743ae168 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:31.991800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4ce757-72bb-4bac-84af-05c6743ae168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:32.003058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ce757-72bb-4bac-84af-05c6743ae168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:32.004526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4ce757-72bb-4bac-84af-05c6743ae168 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-4c4ce757-72bb-4bac-84af-05c6743ae168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:32.020205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:32.021394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:32.022358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:32.022358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:32.022358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:32.022358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 637/1256] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 13 06:51:32.027455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef4339-03d9-4174-800c-31666039d34d None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:32.122214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:32.135977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ef4339-03d9-4174-800c-31666039d34d None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:32.177837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335361a5-4292-4d33-8e69-68f1f6338f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:32.178645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335361a5-4292-4d33-8e69-68f1f6338f93 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:32.178941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335361a5-4292-4d33-8e69-68f1f6338f93 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:32.179389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335361a5-4292-4d33-8e69-68f1f6338f93 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:32.180062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335361a5-4292-4d33-8e69-68f1f6338f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:32.180362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335361a5-4292-4d33-8e69-68f1f6338f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:32.181788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335361a5-4292-4d33-8e69-68f1f6338f93 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:32.194901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 620/1257] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:32.200761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ce757-72bb-4bac-84af-05c6743ae168 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:32.316068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4ce757-72bb-4bac-84af-05c6743ae168 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:32.352028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e7267-700f-43e9-be27-ef64c19cde14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:32.352658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e7267-700f-43e9-be27-ef64c19cde14 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:32.352939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e7267-700f-43e9-be27-ef64c19cde14 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:32.353211 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57e7267-700f-43e9-be27-ef64c19cde14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:32.360707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e7267-700f-43e9-be27-ef64c19cde14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:32.362204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57e7267-700f-43e9-be27-ef64c19cde14 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-d57e7267-700f-43e9-be27-ef64c19cde14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:51:32.375050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:32.376155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:32.377106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:51:32.377106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:51:32.377106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:51:32.377106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:51:32.377106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:51:32.377106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:51:32.377106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:32.377106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:32.377106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 638/1258] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:51:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:32.382372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335361a5-4292-4d33-8e69-68f1f6338f93 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:32.495494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335361a5-4292-4d33-8e69-68f1f6338f93 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:32.532545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b82d595-665f-4877-b721-7520142d6930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:32.533182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82d595-665f-4877-b721-7520142d6930 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:32.533460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82d595-665f-4877-b721-7520142d6930 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:32.533733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b82d595-665f-4877-b721-7520142d6930 None admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:32.534476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82d595-665f-4877-b721-7520142d6930 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:32.534784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82d595-665f-4877-b721-7520142d6930 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:32.536261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b82d595-665f-4877-b721-7520142d6930 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:32.555899 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 621/1259] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:51:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:32.557242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e7267-700f-43e9-be27-ef64c19cde14 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:32.586461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:32.674524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57e7267-700f-43e9-be27-ef64c19cde14 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:32.713686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbb8bf5-8a0c-4e06-9d8c-2b4bee70e2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:32.714333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8bf5-8a0c-4e06-9d8c-2b4bee70e2cb None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:32.714613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8bf5-8a0c-4e06-9d8c-2b4bee70e2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:32.714967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbb8bf5-8a0c-4e06-9d8c-2b4bee70e2cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:32.716364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8bf5-8a0c-4e06-9d8c-2b4bee70e2cb None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=e8202ad6007246078c4d25365dc5a0f5) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:32.716676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8bf5-8a0c-4e06-9d8c-2b4bee70e2cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:32.718173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbb8bf5-8a0c-4e06-9d8c-2b4bee70e2cb None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:32.724617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 639/1260] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:51:32 2022] GET /identity/v3/role_assignments?user.id=e8202ad6007246078c4d25365dc5a0f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:34.897310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc887d-3229-4001-8d11-e7629d9e62d6 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:34.898191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc887d-3229-4001-8d11-e7629d9e62d6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:34.898191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fc887d-3229-4001-8d11-e7629d9e62d6 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:34.900292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 622/1261] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:34.906397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2af5f-037c-487d-8a6d-1f15be40a5ec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:34.906829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2af5f-037c-487d-8a6d-1f15be40a5ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:34.906946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2af5f-037c-487d-8a6d-1f15be40a5ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:34.927449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9cd2af5f-037c-487d-8a6d-1f15be40a5ec None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:34.928630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9cd2af5f-037c-487d-8a6d-1f15be40a5ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:35.003964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 640/1262] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:34 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:51:35.277252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc887d-3229-4001-8d11-e7629d9e62d6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:35.293949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:35.401974 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fc887d-3229-4001-8d11-e7629d9e62d6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:35.437013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5b8ba-ae0c-4235-b4d2-08bd5a490428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:35.437644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5b8ba-ae0c-4235-b4d2-08bd5a490428 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:35.437924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5b8ba-ae0c-4235-b4d2-08bd5a490428 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:35.438235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5b8ba-ae0c-4235-b4d2-08bd5a490428 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:35.448570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a5b8ba-ae0c-4235-b4d2-08bd5a490428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-e0a5b8ba-ae0c-4235-b4d2-08bd5a490428 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:51:35.453527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:35.454660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:35.455879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 623/1263] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:51:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:35.461028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2af5f-037c-487d-8a6d-1f15be40a5ec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:35.479626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:35.564704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd2af5f-037c-487d-8a6d-1f15be40a5ec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:35.603121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080600a-30de-4855-bc82-adc444c8f8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:35.604046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080600a-30de-4855-bc82-adc444c8f8d1 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:35.604493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080600a-30de-4855-bc82-adc444c8f8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:35.604866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f080600a-30de-4855-bc82-adc444c8f8d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:35.605594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080600a-30de-4855-bc82-adc444c8f8d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:35.605895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080600a-30de-4855-bc82-adc444c8f8d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:35.607468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f080600a-30de-4855-bc82-adc444c8f8d1 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:35.620102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 641/1264] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:51:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:35.626067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5b8ba-ae0c-4235-b4d2-08bd5a490428 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:35.694618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:35.731962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5b8ba-ae0c-4235-b4d2-08bd5a490428 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:35.767432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdcc81-e75a-4b69-85d5-c486574197af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:35.768058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdcc81-e75a-4b69-85d5-c486574197af None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:35.768332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdcc81-e75a-4b69-85d5-c486574197af None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:35.768597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43cdcc81-e75a-4b69-85d5-c486574197af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:35.779241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdcc81-e75a-4b69-85d5-c486574197af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:35.780686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43cdcc81-e75a-4b69-85d5-c486574197af None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-43cdcc81-e75a-4b69-85d5-c486574197af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:35.796532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:35.797621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:35.798718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:35.798718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:35.798718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:35.798718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 624/1265] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:35.803803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080600a-30de-4855-bc82-adc444c8f8d1 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:35.891915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:35.919226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f080600a-30de-4855-bc82-adc444c8f8d1 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:35.954593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759b001-5ffb-43a5-86fe-7b33b2785a74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:35.955205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b001-5ffb-43a5-86fe-7b33b2785a74 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:35.955491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b001-5ffb-43a5-86fe-7b33b2785a74 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:35.955759 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0759b001-5ffb-43a5-86fe-7b33b2785a74 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:35.956398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759b001-5ffb-43a5-86fe-7b33b2785a74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:35.956700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759b001-5ffb-43a5-86fe-7b33b2785a74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:35.958112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0759b001-5ffb-43a5-86fe-7b33b2785a74 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:35.969918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 642/1266] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:35.975992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdcc81-e75a-4b69-85d5-c486574197af None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:36.077112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43cdcc81-e75a-4b69-85d5-c486574197af None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:36.096998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:36.114965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ab623-0e65-4ddd-a56b-b10ad67f7ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:36.115591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ab623-0e65-4ddd-a56b-b10ad67f7ccb None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:36.115864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ab623-0e65-4ddd-a56b-b10ad67f7ccb None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:36.116135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ab623-0e65-4ddd-a56b-b10ad67f7ccb None admin] PATH_INFO: `/v3/users/placement` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:36.124791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ab623-0e65-4ddd-a56b-b10ad67f7ccb None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:36.126263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ab623-0e65-4ddd-a56b-b10ad67f7ccb None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-f49ab623-0e65-4ddd-a56b-b10ad67f7ccb None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:51:36.137599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:36.138781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:36.139996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 625/1267] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:36.145389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759b001-5ffb-43a5-86fe-7b33b2785a74 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:36.272026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0759b001-5ffb-43a5-86fe-7b33b2785a74 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:36.312866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e297449-84fb-41be-9f85-a3cb07d42d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:36.313837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e297449-84fb-41be-9f85-a3cb07d42d26 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:36.314347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e297449-84fb-41be-9f85-a3cb07d42d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:36.314802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e297449-84fb-41be-9f85-a3cb07d42d26 None admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:36.315986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e297449-84fb-41be-9f85-a3cb07d42d26 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:36.316529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e297449-84fb-41be-9f85-a3cb07d42d26 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:36.318918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e297449-84fb-41be-9f85-a3cb07d42d26 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:36.333663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:36.345735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 643/1268] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Sep 13 06:51:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:36.351885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ab623-0e65-4ddd-a56b-b10ad67f7ccb None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:36.461215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ab623-0e65-4ddd-a56b-b10ad67f7ccb None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:36.495259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36722e-fe05-46f8-be81-efac181440b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:36.495866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36722e-fe05-46f8-be81-efac181440b6 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:36.496273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36722e-fe05-46f8-be81-efac181440b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:36.496446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f36722e-fe05-46f8-be81-efac181440b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:36.507048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36722e-fe05-46f8-be81-efac181440b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:36.508485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f36722e-fe05-46f8-be81-efac181440b6 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-1f36722e-fe05-46f8-be81-efac181440b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:36.522847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:36.523888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:36.524961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:36.524961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:36.524961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:36.524961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 626/1269] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:36.530019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e297449-84fb-41be-9f85-a3cb07d42d26 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:36.636268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e297449-84fb-41be-9f85-a3cb07d42d26 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:36.672416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480ec4e-f165-481d-a294-e8cedeb71a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:36.673024 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480ec4e-f165-481d-a294-e8cedeb71a77 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:36.673296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480ec4e-f165-481d-a294-e8cedeb71a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:36.673568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3480ec4e-f165-481d-a294-e8cedeb71a77 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:36.674240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480ec4e-f165-481d-a294-e8cedeb71a77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:36.674541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480ec4e-f165-481d-a294-e8cedeb71a77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:36.675924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3480ec4e-f165-481d-a294-e8cedeb71a77 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:36.688158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 644/1270] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:36.697359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36722e-fe05-46f8-be81-efac181440b6 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:36.730200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:36.808215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f36722e-fe05-46f8-be81-efac181440b6 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:36.845396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0a710-32cf-4407-b88c-2f7276c13ee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:36.846010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0a710-32cf-4407-b88c-2f7276c13ee3 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:36.846318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0a710-32cf-4407-b88c-2f7276c13ee3 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:36.846587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c0a710-32cf-4407-b88c-2f7276c13ee3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:36.854111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0a710-32cf-4407-b88c-2f7276c13ee3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:36.855703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5c0a710-32cf-4407-b88c-2f7276c13ee3 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-a5c0a710-32cf-4407-b88c-2f7276c13ee3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:51:36.867086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:36.868100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:36.869272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:51:36.869272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:51:36.869272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:51:36.869272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:51:36.869272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:51:36.869272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:51:36.869272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:36.869272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:36.870291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 627/1271] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:51:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:36.874697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480ec4e-f165-481d-a294-e8cedeb71a77 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:36.945441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:36.998968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3480ec4e-f165-481d-a294-e8cedeb71a77 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:37.037385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40788cbd-e29f-41a7-affb-d0c2d3c0270d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:37.037987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40788cbd-e29f-41a7-affb-d0c2d3c0270d None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:37.038296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40788cbd-e29f-41a7-affb-d0c2d3c0270d None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:37.038567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40788cbd-e29f-41a7-affb-d0c2d3c0270d None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:37.039252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40788cbd-e29f-41a7-affb-d0c2d3c0270d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:37.039556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40788cbd-e29f-41a7-affb-d0c2d3c0270d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:37.040997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40788cbd-e29f-41a7-affb-d0c2d3c0270d None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:37.054346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 645/1272] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:51:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:37.061174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0a710-32cf-4407-b88c-2f7276c13ee3 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:37.178250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:37.190327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c0a710-32cf-4407-b88c-2f7276c13ee3 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:37.226637 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c678cf5-4d61-4789-a97b-f34e2277cdc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:37.227296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c678cf5-4d61-4789-a97b-f34e2277cdc9 None admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:37.227589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c678cf5-4d61-4789-a97b-f34e2277cdc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:37.227878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c678cf5-4d61-4789-a97b-f34e2277cdc9 None admin] PATH_INFO: `/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/e8202ad6007246078c4d25365dc5a0f5/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:37.229769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c678cf5-4d61-4789-a97b-f34e2277cdc9 None admin] RBAC: Authorizing `identity:create_grant(project_id=d05778d5266247d29f296ce5c9792b7e, user_id=e8202ad6007246078c4d25365dc5a0f5, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:37.231357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c678cf5-4d61-4789-a97b-f34e2277cdc9 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:37.249833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 628/1273] 10.222.0.25 () {58 vars in 1439 bytes} [Tue Sep 13 06:51:37 2022] PUT /identity/v3/projects/d05778d5266247d29f296ce5c9792b7e/users/e8202ad6007246078c4d25365dc5a0f5/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:51:39.368691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c091c12-1755-4e24-b185-26faf38b9560 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:39.369576 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c091c12-1755-4e24-b185-26faf38b9560 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:39.369576 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c091c12-1755-4e24-b185-26faf38b9560 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:39.371718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 646/1274] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:39.377131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c2e79-4cd1-4cd8-9498-2c6347221df6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:39.377586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c2e79-4cd1-4cd8-9498-2c6347221df6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:39.377677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6c2e79-4cd1-4cd8-9498-2c6347221df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:39.397852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9c6c2e79-4cd1-4cd8-9498-2c6347221df6 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:39.399076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9c6c2e79-4cd1-4cd8-9498-2c6347221df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:39.470640 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 629/1275] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 13 06:51:39.748276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c091c12-1755-4e24-b185-26faf38b9560 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:39.873081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c091c12-1755-4e24-b185-26faf38b9560 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:39.892069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:39.912043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e969d-f252-49db-890f-e0260e79c92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:39.912685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e969d-f252-49db-890f-e0260e79c92f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:39.912963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e969d-f252-49db-890f-e0260e79c92f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:39.913234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6e969d-f252-49db-890f-e0260e79c92f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:39.923818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6e969d-f252-49db-890f-e0260e79c92f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-1f6e969d-f252-49db-890f-e0260e79c92f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 13 06:51:39.928675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:39.929774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:39.930810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 647/1276] 10.222.0.25 () {56 vars in 1096 bytes} [Tue Sep 13 06:51:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:39.938275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c2e79-4cd1-4cd8-9498-2c6347221df6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:40.021064 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:40.042209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6c2e79-4cd1-4cd8-9498-2c6347221df6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:40.081954 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1b40c-7e03-4aae-910f-e4094e2af540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:40.082582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1b40c-7e03-4aae-910f-e4094e2af540 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:40.082849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1b40c-7e03-4aae-910f-e4094e2af540 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:40.083113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e1b40c-7e03-4aae-910f-e4094e2af540 None admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:40.083770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1b40c-7e03-4aae-910f-e4094e2af540 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:40.084063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1b40c-7e03-4aae-910f-e4094e2af540 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:40.085437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e1b40c-7e03-4aae-910f-e4094e2af540 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:40.094184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 630/1277] 10.222.0.25 () {56 vars in 1099 bytes} [Tue Sep 13 06:51:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 06:51:40.100497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e969d-f252-49db-890f-e0260e79c92f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:40.213966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6e969d-f252-49db-890f-e0260e79c92f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:40.257070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63253fe-da25-44ac-ba6f-eec899b924f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:40.257956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63253fe-da25-44ac-ba6f-eec899b924f3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:40.258359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63253fe-da25-44ac-ba6f-eec899b924f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:40.258815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63253fe-da25-44ac-ba6f-eec899b924f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:40.272883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63253fe-da25-44ac-ba6f-eec899b924f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:40.275074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63253fe-da25-44ac-ba6f-eec899b924f3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-c63253fe-da25-44ac-ba6f-eec899b924f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:40.295439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:40.296654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:40.297764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:40.297764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:40.297764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:40.297764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 648/1278] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:40.303481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1b40c-7e03-4aae-910f-e4094e2af540 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:40.407756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e1b40c-7e03-4aae-910f-e4094e2af540 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:40.446559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2239bd-8189-442d-aa65-e1f4254c89b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:40.447165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2239bd-8189-442d-aa65-e1f4254c89b0 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:40.447437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2239bd-8189-442d-aa65-e1f4254c89b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:40.447706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2239bd-8189-442d-aa65-e1f4254c89b0 None admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:40.448333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2239bd-8189-442d-aa65-e1f4254c89b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:40.448625 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2239bd-8189-442d-aa65-e1f4254c89b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:40.450192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2239bd-8189-442d-aa65-e1f4254c89b0 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:40.461778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 631/1279] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:40.467980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63253fe-da25-44ac-ba6f-eec899b924f3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:40.503983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:40.587319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63253fe-da25-44ac-ba6f-eec899b924f3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:40.626196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be6477-2543-4729-a930-b8bd2a51817f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:40.626856 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be6477-2543-4729-a930-b8bd2a51817f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:40.627129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be6477-2543-4729-a930-b8bd2a51817f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:40.627403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8be6477-2543-4729-a930-b8bd2a51817f None admin] PATH_INFO: `/v3/users/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:40.635072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be6477-2543-4729-a930-b8bd2a51817f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:40.636518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8be6477-2543-4729-a930-b8bd2a51817f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e8be6477-2543-4729-a930-b8bd2a51817f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 13 06:51:40.647880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:40.648957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:40.650251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 649/1280] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:40.655779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2239bd-8189-442d-aa65-e1f4254c89b0 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:40.660039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:40.763661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2239bd-8189-442d-aa65-e1f4254c89b0 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:40.804175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dd740-a735-4e43-a937-6a435881e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:40.804794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd740-a735-4e43-a937-6a435881e2dd None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:40.805074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd740-a735-4e43-a937-6a435881e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:40.805344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59dd740-a735-4e43-a937-6a435881e2dd None admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:40.806136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dd740-a735-4e43-a937-6a435881e2dd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:40.806415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dd740-a735-4e43-a937-6a435881e2dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:40.808191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59dd740-a735-4e43-a937-6a435881e2dd None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:40.824874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 632/1281] 10.222.0.25 () {56 vars in 1143 bytes} [Tue Sep 13 06:51:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:40.833475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be6477-2543-4729-a930-b8bd2a51817f None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:40.929305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:40.940284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8be6477-2543-4729-a930-b8bd2a51817f None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:40.977337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da596e6-361b-4378-81de-a199e055f1f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:40.977946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da596e6-361b-4378-81de-a199e055f1f3 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:40.978258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da596e6-361b-4378-81de-a199e055f1f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:40.978525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da596e6-361b-4378-81de-a199e055f1f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:41.048270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da596e6-361b-4378-81de-a199e055f1f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:41.049733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da596e6-361b-4378-81de-a199e055f1f3 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:41.144246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:41.162390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 650/1282] 10.222.0.25 () {58 vars in 1282 bytes} [Tue Sep 13 06:51:40 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 13 06:51:41.170545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dd740-a735-4e43-a937-6a435881e2dd None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:41.240055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:41.281235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59dd740-a735-4e43-a937-6a435881e2dd None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:41.317546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d22af-527e-41fe-aa4d-48ea8e2780bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:41.318196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d22af-527e-41fe-aa4d-48ea8e2780bf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:41.318476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d22af-527e-41fe-aa4d-48ea8e2780bf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:41.318716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9d22af-527e-41fe-aa4d-48ea8e2780bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:41.330863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d22af-527e-41fe-aa4d-48ea8e2780bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:41.332865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9d22af-527e-41fe-aa4d-48ea8e2780bf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-af9d22af-527e-41fe-aa4d-48ea8e2780bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 13 06:51:41.351597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:41.352687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 13 06:51:41.353824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 13 06:51:41.353824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 13 06:51:41.353824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:41.353824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 633/1283] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:51:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:41.362420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da596e6-361b-4378-81de-a199e055f1f3 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:41.470670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da596e6-361b-4378-81de-a199e055f1f3 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:41.505943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a283-00dc-49a0-855c-f6092de51422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:41.506502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a283-00dc-49a0-855c-f6092de51422 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:41.506778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a283-00dc-49a0-855c-f6092de51422 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:41.507045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a9a283-00dc-49a0-855c-f6092de51422 None admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:41.507679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9a283-00dc-49a0-855c-f6092de51422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:41.507979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9a283-00dc-49a0-855c-f6092de51422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:41.509381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a9a283-00dc-49a0-855c-f6092de51422 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:41.520636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 651/1284] 10.222.0.25 () {56 vars in 1109 bytes} [Tue Sep 13 06:51:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:41.526724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d22af-527e-41fe-aa4d-48ea8e2780bf None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:41.635660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9d22af-527e-41fe-aa4d-48ea8e2780bf None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:41.650271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:41.673389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1f47c-6154-4c44-899b-c901a8fd2339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:41.674015 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1f47c-6154-4c44-899b-c901a8fd2339 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:41.674326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1f47c-6154-4c44-899b-c901a8fd2339 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:41.674595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b1f47c-6154-4c44-899b-c901a8fd2339 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:41.681907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1f47c-6154-4c44-899b-c901a8fd2339 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:41.683933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b1f47c-6154-4c44-899b-c901a8fd2339 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application [None req-55b1f47c-6154-4c44-899b-c901a8fd2339 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 13 06:51:41.696146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:41.697162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:41.702172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 13 06:51:41.702172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 13 06:51:41.702172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 13 06:51:41.702172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 13 06:51:41.702172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 13 06:51:41.702172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 13 06:51:41.702172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 13 06:51:41.702172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: ERROR keystone.server.flask.application Sep 13 06:51:41.702172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 634/1285] 10.222.0.25 () {56 vars in 1111 bytes} [Tue Sep 13 06:51:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:41.705729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a283-00dc-49a0-855c-f6092de51422 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:41.765465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:41.822665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a9a283-00dc-49a0-855c-f6092de51422 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:41.857469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c23739-240a-48d5-b67c-4262a7ea428f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:41.858070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c23739-240a-48d5-b67c-4262a7ea428f None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:41.858375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c23739-240a-48d5-b67c-4262a7ea428f None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:41.858649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c23739-240a-48d5-b67c-4262a7ea428f None admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:41.859341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c23739-240a-48d5-b67c-4262a7ea428f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:41.859646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c23739-240a-48d5-b67c-4262a7ea428f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:41.861078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c23739-240a-48d5-b67c-4262a7ea428f None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:41.873718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 652/1286] 10.222.0.25 () {56 vars in 1148 bytes} [Tue Sep 13 06:51:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:41.880288 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1f47c-6154-4c44-899b-c901a8fd2339 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:41.983771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b1f47c-6154-4c44-899b-c901a8fd2339 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:42.019055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92600a78-6c93-40ba-80ea-d1fb923a3cfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:42.019583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92600a78-6c93-40ba-80ea-d1fb923a3cfa None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:42.019861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92600a78-6c93-40ba-80ea-d1fb923a3cfa None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:42.020129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92600a78-6c93-40ba-80ea-d1fb923a3cfa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:42.020850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92600a78-6c93-40ba-80ea-d1fb923a3cfa None admin] RBAC: Adding query filter params (role.id=756f3eabecc842a0aec6d2e8a9ce5de9, scope.project.id=d05778d5266247d29f296ce5c9792b7e, user.id=e8202ad6007246078c4d25365dc5a0f5) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:42.021156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92600a78-6c93-40ba-80ea-d1fb923a3cfa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:42.022631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92600a78-6c93-40ba-80ea-d1fb923a3cfa None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:42.028464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 635/1287] 10.222.0.25 () {56 vars in 1374 bytes} [Tue Sep 13 06:51:41 2022] GET /identity/v3/role_assignments?user.id=e8202ad6007246078c4d25365dc5a0f5&scope.project.id=d05778d5266247d29f296ce5c9792b7e&role.id=756f3eabecc842a0aec6d2e8a9ce5de9 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:44.170670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6baa0c-921a-4787-85ab-897db744dbe5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:44.171552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6baa0c-921a-4787-85ab-897db744dbe5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:44.171552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6baa0c-921a-4787-85ab-897db744dbe5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:44.173683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 653/1288] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:44.180082 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622bae9-b3f8-4d8c-9209-d1678dcdaef3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:44.180800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622bae9-b3f8-4d8c-9209-d1678dcdaef3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:44.180998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f622bae9-b3f8-4d8c-9209-d1678dcdaef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:44.206979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f622bae9-b3f8-4d8c-9209-d1678dcdaef3 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:44.208175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f622bae9-b3f8-4d8c-9209-d1678dcdaef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:44.243428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:44.289986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 636/1289] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:51:44.553768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6baa0c-921a-4787-85ab-897db744dbe5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:44.664304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6baa0c-921a-4787-85ab-897db744dbe5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:44.675037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:44.701895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9294bc-7ac4-46f2-a866-0d7361e4e4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:44.702556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9294bc-7ac4-46f2-a866-0d7361e4e4bc None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:44.702834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9294bc-7ac4-46f2-a866-0d7361e4e4bc None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:44.703104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9294bc-7ac4-46f2-a866-0d7361e4e4bc None admin] PATH_INFO: `/v3/services/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:44.703691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9294bc-7ac4-46f2-a866-0d7361e4e4bc None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:44.705081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9294bc-7ac4-46f2-a866-0d7361e4e4bc None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ba9294bc-7ac4-46f2-a866-0d7361e4e4bc None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:44.714481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:44.715764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:44.716900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 654/1290] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:51:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:44.722521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622bae9-b3f8-4d8c-9209-d1678dcdaef3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:44.841773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f622bae9-b3f8-4d8c-9209-d1678dcdaef3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:44.881340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0f40b-dcf4-4e2b-a5c3-16c42966883f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:44.881945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0f40b-dcf4-4e2b-a5c3-16c42966883f None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:44.882248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0f40b-dcf4-4e2b-a5c3-16c42966883f None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:44.882521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0f40b-dcf4-4e2b-a5c3-16c42966883f None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:44.883142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0f40b-dcf4-4e2b-a5c3-16c42966883f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:44.883440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0f40b-dcf4-4e2b-a5c3-16c42966883f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:44.884821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0f40b-dcf4-4e2b-a5c3-16c42966883f None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:44.890571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 637/1291] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Sep 13 06:51:44 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:44.896708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9294bc-7ac4-46f2-a866-0d7361e4e4bc None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:45.020023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9294bc-7ac4-46f2-a866-0d7361e4e4bc None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:45.055775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67839c6c-4c7b-4010-bd1e-aa40580e94bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:45.056361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67839c6c-4c7b-4010-bd1e-aa40580e94bd None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:45.056629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67839c6c-4c7b-4010-bd1e-aa40580e94bd None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:45.056898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67839c6c-4c7b-4010-bd1e-aa40580e94bd None admin] PATH_INFO: `/v3/services` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:45.057519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67839c6c-4c7b-4010-bd1e-aa40580e94bd None admin] RBAC: Adding query filter params (type=placement) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:45.057816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67839c6c-4c7b-4010-bd1e-aa40580e94bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:45.059765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67839c6c-4c7b-4010-bd1e-aa40580e94bd None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:45.064396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 655/1292] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Sep 13 06:51:44 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:47.205256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94958ee2-baba-4c6c-8d72-e89a9d35019c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:47.206168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94958ee2-baba-4c6c-8d72-e89a9d35019c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:47.206168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94958ee2-baba-4c6c-8d72-e89a9d35019c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:47.208359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 638/1293] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:47.214829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf8ee5-96a0-4ac9-9e02-c0eba6aaac92 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:47.215381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf8ee5-96a0-4ac9-9e02-c0eba6aaac92 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:47.215742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cf8ee5-96a0-4ac9-9e02-c0eba6aaac92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:47.242999 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-70cf8ee5-96a0-4ac9-9e02-c0eba6aaac92 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:47.244189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-70cf8ee5-96a0-4ac9-9e02-c0eba6aaac92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:47.284830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:47.331187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 656/1294] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:51:47.599747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94958ee2-baba-4c6c-8d72-e89a9d35019c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:47.640166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:47.716897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94958ee2-baba-4c6c-8d72-e89a9d35019c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:47.756173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a9f4bf-c664-4329-90d3-402e8259c33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:47.756768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9f4bf-c664-4329-90d3-402e8259c33a None admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:47.757052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9f4bf-c664-4329-90d3-402e8259c33a None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:47.757324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a9f4bf-c664-4329-90d3-402e8259c33a None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:47.757979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9f4bf-c664-4329-90d3-402e8259c33a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:47.759392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a9f4bf-c664-4329-90d3-402e8259c33a None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:47.777070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 639/1295] 10.222.0.25 () {60 vars in 1141 bytes} [Tue Sep 13 06:51:47 2022] POST /identity/v3/services => generated 246 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:51:49.926255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454435bd-ec3e-4060-9de2-b94a0e9d2f68 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:49.927101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454435bd-ec3e-4060-9de2-b94a0e9d2f68 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:49.927101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454435bd-ec3e-4060-9de2-b94a0e9d2f68 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:49.929124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 657/1296] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:49.935571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd488d-ee96-4511-b331-bb0bfc7382e0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:49.936053 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd488d-ee96-4511-b331-bb0bfc7382e0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:49.936154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd488d-ee96-4511-b331-bb0bfc7382e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:49.957156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-07cd488d-ee96-4511-b331-bb0bfc7382e0 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:49.958379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-07cd488d-ee96-4511-b331-bb0bfc7382e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:50.048160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 640/1297] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:49 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:51:50.323258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454435bd-ec3e-4060-9de2-b94a0e9d2f68 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:50.436766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454435bd-ec3e-4060-9de2-b94a0e9d2f68 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:50.470905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4c829-75a6-4fbb-a31c-aa6b70fa5290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:50.471538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4c829-75a6-4fbb-a31c-aa6b70fa5290 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:50.471823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4c829-75a6-4fbb-a31c-aa6b70fa5290 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:50.472105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade4c829-75a6-4fbb-a31c-aa6b70fa5290 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:50.472685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4c829-75a6-4fbb-a31c-aa6b70fa5290 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:50.474074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade4c829-75a6-4fbb-a31c-aa6b70fa5290 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-ade4c829-75a6-4fbb-a31c-aa6b70fa5290 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:50.483879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:50.485086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:50.486612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 658/1298] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:51:50 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:50.493740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd488d-ee96-4511-b331-bb0bfc7382e0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:50.513382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:50.597953 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd488d-ee96-4511-b331-bb0bfc7382e0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:50.634118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb93ea-3cd6-455c-8245-537af08cb352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:50.634759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb93ea-3cd6-455c-8245-537af08cb352 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:50.635038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb93ea-3cd6-455c-8245-537af08cb352 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:50.635311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fb93ea-3cd6-455c-8245-537af08cb352 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:50.635939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb93ea-3cd6-455c-8245-537af08cb352 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:50.636243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb93ea-3cd6-455c-8245-537af08cb352 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:50.637635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fb93ea-3cd6-455c-8245-537af08cb352 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:50.643262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 641/1299] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Sep 13 06:51:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:50.652252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4c829-75a6-4fbb-a31c-aa6b70fa5290 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:50.685376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:50.755628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade4c829-75a6-4fbb-a31c-aa6b70fa5290 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:50.791762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe4806-5577-4ef2-b744-78152c333582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:50.792365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe4806-5577-4ef2-b744-78152c333582 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:50.792640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe4806-5577-4ef2-b744-78152c333582 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:50.792908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe4806-5577-4ef2-b744-78152c333582 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:50.793605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe4806-5577-4ef2-b744-78152c333582 None admin] RBAC: Adding query filter params (interface=public, service_id=45ddb08585404bfeb7a20756356fa365, region_id=RegionOne) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:50.793885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe4806-5577-4ef2-b744-78152c333582 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:50.795454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe4806-5577-4ef2-b744-78152c333582 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:50.800712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 659/1300] 10.222.0.25 () {56 vars in 1251 bytes} [Tue Sep 13 06:51:50 2022] GET /identity/v3/endpoints?service_id=45ddb08585404bfeb7a20756356fa365&interface=public®ion_id=RegionOne => generated 194 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:50.806705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb93ea-3cd6-455c-8245-537af08cb352 None admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:50.899599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:50.918484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fb93ea-3cd6-455c-8245-537af08cb352 None admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:50.955467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ac4d5b-e87e-42f4-8788-75d67be9bc10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:50.956145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac4d5b-e87e-42f4-8788-75d67be9bc10 None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:50.956334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac4d5b-e87e-42f4-8788-75d67be9bc10 None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:50.956605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ac4d5b-e87e-42f4-8788-75d67be9bc10 None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:50.957208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac4d5b-e87e-42f4-8788-75d67be9bc10 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:50.958617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ac4d5b-e87e-42f4-8788-75d67be9bc10 None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:50.965864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 642/1301] 10.222.0.25 () {56 vars in 1087 bytes} [Tue Sep 13 06:51:50 2022] GET /identity/v3/services => generated 2191 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 06:51:53.109306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc471d3-2850-4742-b4bd-dd365e1d1355 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:53.110308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc471d3-2850-4742-b4bd-dd365e1d1355 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:53.110308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc471d3-2850-4742-b4bd-dd365e1d1355 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:53.112384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 660/1302] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:51:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:51:53.118718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf600af-1462-4d0c-9bfa-e8185c89cbe1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:53.119474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf600af-1462-4d0c-9bfa-e8185c89cbe1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:53.119616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf600af-1462-4d0c-9bfa-e8185c89cbe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:53.146341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9bf600af-1462-4d0c-9bfa-e8185c89cbe1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:51:53.147531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9bf600af-1462-4d0c-9bfa-e8185c89cbe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:51:53.221572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 643/1303] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:51:53 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 13 06:51:53.494722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc471d3-2850-4742-b4bd-dd365e1d1355 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:53.612690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:53.620621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc471d3-2850-4742-b4bd-dd365e1d1355 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:53.657291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e066cf-c182-48c7-a9d9-c46347c34088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:53.657915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e066cf-c182-48c7-a9d9-c46347c34088 None admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:53.658231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e066cf-c182-48c7-a9d9-c46347c34088 None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:53.658509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e066cf-c182-48c7-a9d9-c46347c34088 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:53.659091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e066cf-c182-48c7-a9d9-c46347c34088 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:53.660471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e066cf-c182-48c7-a9d9-c46347c34088 None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application [None req-e5e066cf-c182-48c7-a9d9-c46347c34088 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 13 06:51:53.669534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 13 06:51:53.670657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.get_or_create( Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application with Lock( Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self._enter() Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application return self.creator() Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application created_value = creator( Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: ERROR keystone.server.flask.application Sep 13 06:51:53.671678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 661/1304] 10.222.0.25 () {56 vars in 1117 bytes} [Tue Sep 13 06:51:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 13 06:51:53.679169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf600af-1462-4d0c-9bfa-e8185c89cbe1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:53.756441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:51:53.782663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf600af-1462-4d0c-9bfa-e8185c89cbe1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:53.817360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accd235e-977e-426a-8ba1-22e77e4b4faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:53.817939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd235e-977e-426a-8ba1-22e77e4b4faf None admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:53.818251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd235e-977e-426a-8ba1-22e77e4b4faf None admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:53.818512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accd235e-977e-426a-8ba1-22e77e4b4faf None admin] PATH_INFO: `/v3/services` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:53.819129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd235e-977e-426a-8ba1-22e77e4b4faf None admin] RBAC: Adding query filter params (name=placement) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:51:53.819435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd235e-977e-426a-8ba1-22e77e4b4faf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:53.820805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accd235e-977e-426a-8ba1-22e77e4b4faf None admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:53.826183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 644/1305] 10.222.0.25 () {56 vars in 1116 bytes} [Tue Sep 13 06:51:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:51:53.832215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e066cf-c182-48c7-a9d9-c46347c34088 None admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:51:53.940498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e066cf-c182-48c7-a9d9-c46347c34088 None admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:51:53.976785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17416a17-6829-49ab-9fb9-3be9a7542ade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:51:53.977369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17416a17-6829-49ab-9fb9-3be9a7542ade None admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:51:53.977640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17416a17-6829-49ab-9fb9-3be9a7542ade None admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:51:53.977899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17416a17-6829-49ab-9fb9-3be9a7542ade None admin] PATH_INFO: `/v3/endpoints` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:51:53.978591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17416a17-6829-49ab-9fb9-3be9a7542ade None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:51:53.979963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17416a17-6829-49ab-9fb9-3be9a7542ade None admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:51:54.003041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 662/1306] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:51:53 2022] POST /identity/v3/endpoints => generated 331 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:52:03.822933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=30 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:52:03.998580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:52:59.463746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1d7af-2b33-4c46-ade4-867cf235e532 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:52:59.464349 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1d7af-2b33-4c46-ade4-867cf235e532 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:52:59.464349 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f1d7af-2b33-4c46-ade4-867cf235e532 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:52:59.465837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 645/1307] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:52:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:52:59.471764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af4ddd-47e4-4f60-94e9-088ea5a9caf6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:52:59.472590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af4ddd-47e4-4f60-94e9-088ea5a9caf6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:52:59.473021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9af4ddd-47e4-4f60-94e9-088ea5a9caf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:52:59.626499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a9af4ddd-47e4-4f60-94e9-088ea5a9caf6 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:52:59.628102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a9af4ddd-47e4-4f60-94e9-088ea5a9caf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:52:59.813597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 663/1308] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:52:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:00.109554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621626b7-c906-4d4d-8d65-76a1672036c2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:00.110158 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621626b7-c906-4d4d-8d65-76a1672036c2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:00.110596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621626b7-c906-4d4d-8d65-76a1672036c2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:00.112328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 646/1309] 10.222.0.25 () {54 vars in 965 bytes} [Tue Sep 13 06:53:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:00.118690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51603fbd-c771-4858-ba1e-616ecc140596 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:00.118992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51603fbd-c771-4858-ba1e-616ecc140596 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:00.119269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51603fbd-c771-4858-ba1e-616ecc140596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:00.208731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-51603fbd-c771-4858-ba1e-616ecc140596 None None] MFA Rules not processed for user `f94aa8af227a42a08d8a0e4b7f02b85a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:00.210154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-51603fbd-c771-4858-ba1e-616ecc140596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:00.236960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:00.313114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 664/1310] 10.222.0.25 () {58 vars in 1062 bytes} [Tue Sep 13 06:53:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:00.322331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621626b7-c906-4d4d-8d65-76a1672036c2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:00.324354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-621626b7-c906-4d4d-8d65-76a1672036c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:00.466696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621626b7-c906-4d4d-8d65-76a1672036c2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:00.554703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c0c13-1e53-4fde-a19f-9560c808aebd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:00.555323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c0c13-1e53-4fde-a19f-9560c808aebd service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:00.555598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c0c13-1e53-4fde-a19f-9560c808aebd service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:00.555863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c0c13-1e53-4fde-a19f-9560c808aebd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:00.557586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb8c0c13-1e53-4fde-a19f-9560c808aebd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:00.619674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:00.683360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c0c13-1e53-4fde-a19f-9560c808aebd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:00.684900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c0c13-1e53-4fde-a19f-9560c808aebd service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:00.718610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 647/1311] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:03.150224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da324c0-3f77-4852-a055-cfe3f09e4d39 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:03.150780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da324c0-3f77-4852-a055-cfe3f09e4d39 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:03.150780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da324c0-3f77-4852-a055-cfe3f09e4d39 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:03.152128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 665/1312] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:03.158756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586b52-78bc-431c-b9a1-8ecc180a81cf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:03.159526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586b52-78bc-431c-b9a1-8ecc180a81cf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:03.159711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586b52-78bc-431c-b9a1-8ecc180a81cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:03.249041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-54586b52-78bc-431c-b9a1-8ecc180a81cf None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:03.250289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-54586b52-78bc-431c-b9a1-8ecc180a81cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:03.351170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 648/1313] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:03.633771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da324c0-3f77-4852-a055-cfe3f09e4d39 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:03.635418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7da324c0-3f77-4852-a055-cfe3f09e4d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:03.745949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da324c0-3f77-4852-a055-cfe3f09e4d39 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:03.821808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:03.824193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb433481-fcc2-46f9-955a-60d6f2dc79d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:03.824808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb433481-fcc2-46f9-955a-60d6f2dc79d7 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:03.825087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb433481-fcc2-46f9-955a-60d6f2dc79d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:03.825363 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb433481-fcc2-46f9-955a-60d6f2dc79d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:03.828045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fb433481-fcc2-46f9-955a-60d6f2dc79d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:03.945989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb433481-fcc2-46f9-955a-60d6f2dc79d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:03.947397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb433481-fcc2-46f9-955a-60d6f2dc79d7 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:03.965707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 666/1314] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:06.431867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36236e-691f-4095-b688-29da9b79d8ab None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:06.432828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36236e-691f-4095-b688-29da9b79d8ab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:06.432828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36236e-691f-4095-b688-29da9b79d8ab None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:06.435358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 649/1315] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:06.440832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa344459-a9a4-4105-8ad5-641bd4cfcdde None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:06.441298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa344459-a9a4-4105-8ad5-641bd4cfcdde None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:06.441405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa344459-a9a4-4105-8ad5-641bd4cfcdde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:06.520142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fa344459-a9a4-4105-8ad5-641bd4cfcdde None None] MFA Rules not processed for user `a0286a556e15432dbc2bd4ac3f2e801a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:06.521355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fa344459-a9a4-4105-8ad5-641bd4cfcdde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:06.588302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:06.633998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 667/1316] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:06 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:06.927917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36236e-691f-4095-b688-29da9b79d8ab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:06.930002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-db36236e-691f-4095-b688-29da9b79d8ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:06.955668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:07.088372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36236e-691f-4095-b688-29da9b79d8ab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:07.197739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc75cba-240f-4619-b628-54c2b7d47f88 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:07.198693 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc75cba-240f-4619-b628-54c2b7d47f88 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:07.199129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc75cba-240f-4619-b628-54c2b7d47f88 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:07.199545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc75cba-240f-4619-b628-54c2b7d47f88 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:07.201271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2fc75cba-240f-4619-b628-54c2b7d47f88 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:07.279108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:07.340537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc75cba-240f-4619-b628-54c2b7d47f88 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:07.342376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc75cba-240f-4619-b628-54c2b7d47f88 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:07.370487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 650/1317] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:06 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:10.154527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542caf0-93c0-41b8-a6c6-8f0cd810ff73 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:10.155203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542caf0-93c0-41b8-a6c6-8f0cd810ff73 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:10.155203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542caf0-93c0-41b8-a6c6-8f0cd810ff73 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:10.156536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 668/1318] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:10.163001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa928-c8d1-49b3-b26c-085808c883a7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:10.163735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa928-c8d1-49b3-b26c-085808c883a7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:10.163928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319aa928-c8d1-49b3-b26c-085808c883a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:10.249058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-319aa928-c8d1-49b3-b26c-085808c883a7 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:10.249992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-319aa928-c8d1-49b3-b26c-085808c883a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:10.359234 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 651/1319] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:10.639841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542caf0-93c0-41b8-a6c6-8f0cd810ff73 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:10.641402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c542caf0-93c0-41b8-a6c6-8f0cd810ff73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:10.748203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542caf0-93c0-41b8-a6c6-8f0cd810ff73 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:10.825738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be0f420-ae20-49c7-8baa-810b06f91613 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:10.826408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0f420-ae20-49c7-8baa-810b06f91613 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:10.826664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0f420-ae20-49c7-8baa-810b06f91613 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:10.826956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be0f420-ae20-49c7-8baa-810b06f91613 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:10.828562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8be0f420-ae20-49c7-8baa-810b06f91613 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:10.835739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:10.942164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0f420-ae20-49c7-8baa-810b06f91613 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:10.943523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be0f420-ae20-49c7-8baa-810b06f91613 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:10.966831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 669/1320] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:13.781034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccc55f-5cdc-4264-afda-dcd3d16a027b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:13.781658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccc55f-5cdc-4264-afda-dcd3d16a027b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:13.781658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ccc55f-5cdc-4264-afda-dcd3d16a027b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:13.783220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 652/1321] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:13.788694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1e821-410b-4460-981e-1f7e34108053 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:13.789157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1e821-410b-4460-981e-1f7e34108053 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:13.789272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d1e821-410b-4460-981e-1f7e34108053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:13.862809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-49d1e821-410b-4460-981e-1f7e34108053 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:13.863953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-49d1e821-410b-4460-981e-1f7e34108053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:13.927993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:13.973762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 670/1322] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:14.254444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccc55f-5cdc-4264-afda-dcd3d16a027b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:14.256030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-08ccc55f-5cdc-4264-afda-dcd3d16a027b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:14.277203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:14.368173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ccc55f-5cdc-4264-afda-dcd3d16a027b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:14.445134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d5197e-6156-46e2-8dea-03618198967e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:14.445750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5197e-6156-46e2-8dea-03618198967e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:14.446036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5197e-6156-46e2-8dea-03618198967e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:14.446323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d5197e-6156-46e2-8dea-03618198967e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:14.447919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e3d5197e-6156-46e2-8dea-03618198967e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:14.525311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:14.566948 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5197e-6156-46e2-8dea-03618198967e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:14.568407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d5197e-6156-46e2-8dea-03618198967e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:14.587530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 653/1323] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:16.796189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01347081-a895-49f7-a8ac-1620a07c94e2 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:16.796815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01347081-a895-49f7-a8ac-1620a07c94e2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:16.796815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01347081-a895-49f7-a8ac-1620a07c94e2 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:16.798116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 671/1324] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:16.803826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffe51f-a945-48ef-92ce-4d12fc540602 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:16.804302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffe51f-a945-48ef-92ce-4d12fc540602 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:16.804444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ffe51f-a945-48ef-92ce-4d12fc540602 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:16.874263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-50ffe51f-a945-48ef-92ce-4d12fc540602 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:16.875853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-50ffe51f-a945-48ef-92ce-4d12fc540602 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:16.988734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 654/1325] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:17.266230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01347081-a895-49f7-a8ac-1620a07c94e2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:17.267803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-01347081-a895-49f7-a8ac-1620a07c94e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:17.382540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01347081-a895-49f7-a8ac-1620a07c94e2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:17.459026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272d6072-1de9-494f-ace3-ebcaa3ccb903 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:17.459637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d6072-1de9-494f-ace3-ebcaa3ccb903 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:17.459911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d6072-1de9-494f-ace3-ebcaa3ccb903 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:17.460171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272d6072-1de9-494f-ace3-ebcaa3ccb903 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:17.461758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-272d6072-1de9-494f-ace3-ebcaa3ccb903 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:17.478619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:17.575104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d6072-1de9-494f-ace3-ebcaa3ccb903 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:17.576604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272d6072-1de9-494f-ace3-ebcaa3ccb903 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:17.597859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 672/1326] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:19.806189 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe06252-0cab-4a2a-90ff-89a4eec99143 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:19.806729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe06252-0cab-4a2a-90ff-89a4eec99143 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:19.806729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe06252-0cab-4a2a-90ff-89a4eec99143 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:19.808395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 655/1327] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:19.814106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdb57d-6e81-473e-b62d-c1526c84f249 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:19.814599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdb57d-6e81-473e-b62d-c1526c84f249 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:19.814704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cdb57d-6e81-473e-b62d-c1526c84f249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:19.886703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-79cdb57d-6e81-473e-b62d-c1526c84f249 None None] MFA Rules not processed for user `a0286a556e15432dbc2bd4ac3f2e801a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:19.887957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-79cdb57d-6e81-473e-b62d-c1526c84f249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:19.956270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:19.989440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 673/1328] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:19 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:20.270478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe06252-0cab-4a2a-90ff-89a4eec99143 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:20.272068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ffe06252-0cab-4a2a-90ff-89a4eec99143 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:20.306843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:20.385740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe06252-0cab-4a2a-90ff-89a4eec99143 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:20.462363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcb69d-e68f-403e-a446-7b09cc714a8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:20.462979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb69d-e68f-403e-a446-7b09cc714a8d service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:20.463254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb69d-e68f-403e-a446-7b09cc714a8d service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:20.463520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcb69d-e68f-403e-a446-7b09cc714a8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:20.465111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-11fcb69d-e68f-403e-a446-7b09cc714a8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:20.548799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:20.585756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcb69d-e68f-403e-a446-7b09cc714a8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:20.587369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcb69d-e68f-403e-a446-7b09cc714a8d service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:20.606156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 656/1329] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:22.989622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f1ecd-c52b-4ce9-bb83-8e0d5e29b587 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:22.990590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f1ecd-c52b-4ce9-bb83-8e0d5e29b587 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:22.990590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f1ecd-c52b-4ce9-bb83-8e0d5e29b587 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:22.992613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 674/1330] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:22.999869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610089be-15d8-4a07-b56b-32e4a356f772 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:23.000613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610089be-15d8-4a07-b56b-32e4a356f772 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:23.000613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610089be-15d8-4a07-b56b-32e4a356f772 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:23.082122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-610089be-15d8-4a07-b56b-32e4a356f772 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:23.083412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-610089be-15d8-4a07-b56b-32e4a356f772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:23.188159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 657/1331] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:23.472593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f1ecd-c52b-4ce9-bb83-8e0d5e29b587 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:23.475119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-711f1ecd-c52b-4ce9-bb83-8e0d5e29b587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:23.600058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f1ecd-c52b-4ce9-bb83-8e0d5e29b587 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:23.688754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b973759e-a96e-4a16-ac4f-d8c2c8de5479 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:23.689376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973759e-a96e-4a16-ac4f-d8c2c8de5479 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:23.689657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973759e-a96e-4a16-ac4f-d8c2c8de5479 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:23.689900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b973759e-a96e-4a16-ac4f-d8c2c8de5479 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:23.691570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b973759e-a96e-4a16-ac4f-d8c2c8de5479 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:23.720149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:23.810893 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973759e-a96e-4a16-ac4f-d8c2c8de5479 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:23.812345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b973759e-a96e-4a16-ac4f-d8c2c8de5479 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:23.830955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 675/1332] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:26.044957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b04d9-ca9f-4595-be30-7ecefde7c86c None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:26.045534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b04d9-ca9f-4595-be30-7ecefde7c86c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:26.045629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12b04d9-ca9f-4595-be30-7ecefde7c86c None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:26.047016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 658/1333] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:26.052862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0b115-620a-4f3c-bcc8-826a6d23c782 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:26.053371 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0b115-620a-4f3c-bcc8-826a6d23c782 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:26.053486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0b115-620a-4f3c-bcc8-826a6d23c782 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:26.138911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-23c0b115-620a-4f3c-bcc8-826a6d23c782 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:26.140233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-23c0b115-620a-4f3c-bcc8-826a6d23c782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:26.223622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:26.247144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 676/1334] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:26.531118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b04d9-ca9f-4595-be30-7ecefde7c86c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:26.532677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f12b04d9-ca9f-4595-be30-7ecefde7c86c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:26.580110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:26.646605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12b04d9-ca9f-4595-be30-7ecefde7c86c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:26.725473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca9e96d-8aa4-4732-9206-ecd70f1aaf51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:26.726182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9e96d-8aa4-4732-9206-ecd70f1aaf51 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:26.726488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9e96d-8aa4-4732-9206-ecd70f1aaf51 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:26.726791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca9e96d-8aa4-4732-9206-ecd70f1aaf51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:26.728379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5ca9e96d-8aa4-4732-9206-ecd70f1aaf51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:26.825978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:26.851920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9e96d-8aa4-4732-9206-ecd70f1aaf51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:26.853385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca9e96d-8aa4-4732-9206-ecd70f1aaf51 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:26.871167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 659/1335] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:29.413500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa89d-8682-47e9-ad3f-efd9eaba795b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:29.414415 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa89d-8682-47e9-ad3f-efd9eaba795b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:29.414415 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16aa89d-8682-47e9-ad3f-efd9eaba795b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:29.416545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 677/1336] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:29.425630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e5abf-df87-4b65-b1b3-8961933bbee9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:29.426479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e5abf-df87-4b65-b1b3-8961933bbee9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:29.426630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902e5abf-df87-4b65-b1b3-8961933bbee9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:29.501873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-902e5abf-df87-4b65-b1b3-8961933bbee9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:29.503133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-902e5abf-df87-4b65-b1b3-8961933bbee9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:29.608648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 660/1337] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:29.898721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa89d-8682-47e9-ad3f-efd9eaba795b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:29.901262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b16aa89d-8682-47e9-ad3f-efd9eaba795b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:30.018022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16aa89d-8682-47e9-ad3f-efd9eaba795b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:30.108185 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb4450c-5af6-423b-8f47-32c1b7ab0f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:30.108802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4450c-5af6-423b-8f47-32c1b7ab0f65 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:30.109088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4450c-5af6-423b-8f47-32c1b7ab0f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:30.109362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb4450c-5af6-423b-8f47-32c1b7ab0f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:30.111035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-eeb4450c-5af6-423b-8f47-32c1b7ab0f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:30.148610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:30.226551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4450c-5af6-423b-8f47-32c1b7ab0f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:30.228109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb4450c-5af6-423b-8f47-32c1b7ab0f65 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:30.248144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 678/1338] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:29 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:34.396012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca7f90-2927-4a8e-a8df-e8dd2afb9641 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:34.396577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca7f90-2927-4a8e-a8df-e8dd2afb9641 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:34.396577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca7f90-2927-4a8e-a8df-e8dd2afb9641 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:34.398039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 661/1339] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:34.403247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb6c82-5a6d-4d53-90e6-afb67f3686b4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:34.403702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb6c82-5a6d-4d53-90e6-afb67f3686b4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:34.403798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb6c82-5a6d-4d53-90e6-afb67f3686b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:34.477127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-56eb6c82-5a6d-4d53-90e6-afb67f3686b4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:34.478314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-56eb6c82-5a6d-4d53-90e6-afb67f3686b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:34.570571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:34.583181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 679/1340] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:34.877065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb8310-6016-4598-8c85-2aeedd8646ea None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:34.877353 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb8310-6016-4598-8c85-2aeedd8646ea None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:34.877679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fb8310-6016-4598-8c85-2aeedd8646ea None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:34.879104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 662/1341] 10.222.0.25 () {54 vars in 965 bytes} [Tue Sep 13 06:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:34.885055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff62455-6c04-4e3a-9bf2-f8362fe1a9a4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:34.885342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff62455-6c04-4e3a-9bf2-f8362fe1a9a4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:34.885613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff62455-6c04-4e3a-9bf2-f8362fe1a9a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:34.965945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3ff62455-6c04-4e3a-9bf2-f8362fe1a9a4 None None] MFA Rules not processed for user `f94aa8af227a42a08d8a0e4b7f02b85a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:34.967260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3ff62455-6c04-4e3a-9bf2-f8362fe1a9a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:35.045408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 680/1342] 10.222.0.25 () {58 vars in 1062 bytes} [Tue Sep 13 06:53:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:35.054465 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb8310-6016-4598-8c85-2aeedd8646ea None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:35.056009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-81fb8310-6016-4598-8c85-2aeedd8646ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:35.107599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:35.165115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fb8310-6016-4598-8c85-2aeedd8646ea None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:35.245801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a62c6-b063-4911-a819-77329ae06fd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:35.246475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a62c6-b063-4911-a819-77329ae06fd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:35.246740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a62c6-b063-4911-a819-77329ae06fd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:35.247030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a62c6-b063-4911-a819-77329ae06fd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:35.249510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-056a62c6-b063-4911-a819-77329ae06fd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:35.359021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:35.378220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a62c6-b063-4911-a819-77329ae06fd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:35.379668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a62c6-b063-4911-a819-77329ae06fd7 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:35.400062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 663/1343] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:38.713553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dc900-b5ac-43c8-894c-0bb06736bb2c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:38.714070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dc900-b5ac-43c8-894c-0bb06736bb2c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:38.714195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46dc900-b5ac-43c8-894c-0bb06736bb2c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:38.715538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 681/1344] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:38.721577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5094a2-f4bb-4395-83e1-6fcbf718bcd1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:38.722055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5094a2-f4bb-4395-83e1-6fcbf718bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:38.722183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5094a2-f4bb-4395-83e1-6fcbf718bcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:38.790314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7d5094a2-f4bb-4395-83e1-6fcbf718bcd1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:38.791534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7d5094a2-f4bb-4395-83e1-6fcbf718bcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:38.909910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 664/1345] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:39.199376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dc900-b5ac-43c8-894c-0bb06736bb2c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:39.200944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c46dc900-b5ac-43c8-894c-0bb06736bb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:39.312813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:39.331896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46dc900-b5ac-43c8-894c-0bb06736bb2c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:39.407773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3123a88-da8b-411d-a349-7f0edbe3e073 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:39.408456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3123a88-da8b-411d-a349-7f0edbe3e073 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:39.408746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3123a88-da8b-411d-a349-7f0edbe3e073 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:39.409030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3123a88-da8b-411d-a349-7f0edbe3e073 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:39.410669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e3123a88-da8b-411d-a349-7f0edbe3e073 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:39.523048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3123a88-da8b-411d-a349-7f0edbe3e073 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:39.524522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3123a88-da8b-411d-a349-7f0edbe3e073 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:39.543739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 682/1346] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:39 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:43.948114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d7cd2-3de8-4307-aa62-75871e386960 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:43.948654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d7cd2-3de8-4307-aa62-75871e386960 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:43.948654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88d7cd2-3de8-4307-aa62-75871e386960 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:43.950201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 665/1347] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:43.956156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4636148-c5e6-4554-9af8-24cec541f99f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:43.956632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4636148-c5e6-4554-9af8-24cec541f99f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:43.956728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4636148-c5e6-4554-9af8-24cec541f99f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:43.973375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:44.026012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d4636148-c5e6-4554-9af8-24cec541f99f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:44.027250 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d4636148-c5e6-4554-9af8-24cec541f99f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:44.135629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 683/1348] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:44.421228 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d7cd2-3de8-4307-aa62-75871e386960 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:53:44.423768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a88d7cd2-3de8-4307-aa62-75871e386960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:44.487651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:44.546225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88d7cd2-3de8-4307-aa62-75871e386960 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:53:44.632744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b793ca-182e-46f5-9ff3-016218b57649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:53:44.633367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b793ca-182e-46f5-9ff3-016218b57649 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:44.633646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b793ca-182e-46f5-9ff3-016218b57649 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:44.633920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b793ca-182e-46f5-9ff3-016218b57649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:44.635574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a1b793ca-182e-46f5-9ff3-016218b57649 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:44.752851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:44.757031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b793ca-182e-46f5-9ff3-016218b57649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:53:44.758988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b793ca-182e-46f5-9ff3-016218b57649 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:53:44.778819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 666/1349] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:53:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:53:49.751910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c628747-ec5d-4dab-a502-067607daee68 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:49.752910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c628747-ec5d-4dab-a502-067607daee68 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:49.753414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c628747-ec5d-4dab-a502-067607daee68 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:49.755277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 684/1350] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 06:53:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:49.761969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7767caa-5115-4d37-ad1a-8db4114c2e18 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:49.762857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7767caa-5115-4d37-ad1a-8db4114c2e18 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:49.763349 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7767caa-5115-4d37-ad1a-8db4114c2e18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:49.859536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c7767caa-5115-4d37-ad1a-8db4114c2e18 None None] MFA Rules not processed for user `e8202ad6007246078c4d25365dc5a0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:49.861328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c7767caa-5115-4d37-ad1a-8db4114c2e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:50.031480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 667/1351] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 06:53:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:54.133091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=59 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:54.228720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32790d-ef85-4551-aa85-2ad5c8489779 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:54.229121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32790d-ef85-4551-aa85-2ad5c8489779 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:54.229256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af32790d-ef85-4551-aa85-2ad5c8489779 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:54.230929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 685/1352] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 06:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:54.236164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e0243-7d04-4ff8-8cc9-5b7cb19ade44 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:54.236686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e0243-7d04-4ff8-8cc9-5b7cb19ade44 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:54.236686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123e0243-7d04-4ff8-8cc9-5b7cb19ade44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:54.295010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:54.322560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-123e0243-7d04-4ff8-8cc9-5b7cb19ade44 None None] MFA Rules not processed for user `e8202ad6007246078c4d25365dc5a0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:54.323844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-123e0243-7d04-4ff8-8cc9-5b7cb19ade44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:54.449633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 668/1353] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 06:53:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:55.722555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1da49-6250-4d11-8a72-a1059b574a0b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:55.723104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1da49-6250-4d11-8a72-a1059b574a0b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:55.723104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd1da49-6250-4d11-8a72-a1059b574a0b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:55.724435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 686/1354] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 06:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:55.730579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df020de0-0f83-41ab-8cc4-c14324a247c9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:55.731307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df020de0-0f83-41ab-8cc4-c14324a247c9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:55.731804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df020de0-0f83-41ab-8cc4-c14324a247c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:55.818920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-df020de0-0f83-41ab-8cc4-c14324a247c9 None None] MFA Rules not processed for user `e8202ad6007246078c4d25365dc5a0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:55.820123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-df020de0-0f83-41ab-8cc4-c14324a247c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:55.864339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:53:55.939303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 669/1355] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 06:53:55 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:59.401143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29074-0c1a-4ee1-9389-196fc25d09e1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:59.402105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29074-0c1a-4ee1-9389-196fc25d09e1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:59.405486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab29074-0c1a-4ee1-9389-196fc25d09e1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:59.407102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 687/1356] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:53:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:59.412963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe157c-b508-4e12-92d2-727a428f316b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:59.413472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe157c-b508-4e12-92d2-727a428f316b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:59.413820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebe157c-b508-4e12-92d2-727a428f316b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:59.495426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bebe157c-b508-4e12-92d2-727a428f316b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:53:59.496783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bebe157c-b508-4e12-92d2-727a428f316b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:53:59.602126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 670/1357] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:53:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:53:59.899455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6def6e8-e8bf-447e-87ea-5db747299730 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:59.900234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6def6e8-e8bf-447e-87ea-5db747299730 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:59.900754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6def6e8-e8bf-447e-87ea-5db747299730 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:59.902466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 688/1358] 10.222.0.25 () {54 vars in 962 bytes} [Tue Sep 13 06:53:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:53:59.911078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aaba8-e21e-4c39-adad-ffa4a0e5b855 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:53:59.911848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aaba8-e21e-4c39-adad-ffa4a0e5b855 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:53:59.912331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3aaba8-e21e-4c39-adad-ffa4a0e5b855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:53:59.934359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:00.015687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ac3aaba8-e21e-4c39-adad-ffa4a0e5b855 None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:00.017055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ac3aaba8-e21e-4c39-adad-ffa4a0e5b855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:00.142872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 671/1359] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Sep 13 06:53:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:00.150991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6def6e8-e8bf-447e-87ea-5db747299730 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:00.152618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e6def6e8-e8bf-447e-87ea-5db747299730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:00.310123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6def6e8-e8bf-447e-87ea-5db747299730 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:00.385396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:00.427057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b761ed3-cfba-411c-ba81-0db1454e5844 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:00.427788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b761ed3-cfba-411c-ba81-0db1454e5844 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:00.428181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b761ed3-cfba-411c-ba81-0db1454e5844 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:00.428531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b761ed3-cfba-411c-ba81-0db1454e5844 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:00.430393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1b761ed3-cfba-411c-ba81-0db1454e5844 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:00.535446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b761ed3-cfba-411c-ba81-0db1454e5844 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:00.537246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b761ed3-cfba-411c-ba81-0db1454e5844 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:00.557331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 689/1360] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:02.939331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fcdce-f596-49f5-bd11-283016fbce60 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:02.939919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fcdce-f596-49f5-bd11-283016fbce60 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:02.939919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50fcdce-f596-49f5-bd11-283016fbce60 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:02.941480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 672/1361] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:02.952198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa4e77-39aa-48da-ba2b-4922acfcd9db None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:02.952887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa4e77-39aa-48da-ba2b-4922acfcd9db None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:02.952887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa4e77-39aa-48da-ba2b-4922acfcd9db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:03.027005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-befa4e77-39aa-48da-ba2b-4922acfcd9db None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:03.028733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-befa4e77-39aa-48da-ba2b-4922acfcd9db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:03.071985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:03.137985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 690/1362] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:03.427259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fcdce-f596-49f5-bd11-283016fbce60 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:03.428693 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c50fcdce-f596-49f5-bd11-283016fbce60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:03.504489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:03.616234 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50fcdce-f596-49f5-bd11-283016fbce60 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:03.745008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d19d3-bbf6-4ac9-b603-f36b41bbefb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:03.745611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d19d3-bbf6-4ac9-b603-f36b41bbefb2 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:03.745887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d19d3-bbf6-4ac9-b603-f36b41bbefb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:03.746204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d19d3-bbf6-4ac9-b603-f36b41bbefb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:03.748152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ce9d19d3-bbf6-4ac9-b603-f36b41bbefb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:03.771037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:03.841151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d19d3-bbf6-4ac9-b603-f36b41bbefb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:03.842961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d19d3-bbf6-4ac9-b603-f36b41bbefb2 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:03.861832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 673/1363] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:06.328705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e61376-dade-4756-84f4-987c131440c1 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:06.329274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e61376-dade-4756-84f4-987c131440c1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:06.329274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e61376-dade-4756-84f4-987c131440c1 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:06.330630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 691/1364] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:06.336246 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00600f65-2b14-4d8d-b0e1-5d318188a5ff None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:06.336738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00600f65-2b14-4d8d-b0e1-5d318188a5ff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:06.336925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00600f65-2b14-4d8d-b0e1-5d318188a5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:06.406663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-00600f65-2b14-4d8d-b0e1-5d318188a5ff None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:06.407938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-00600f65-2b14-4d8d-b0e1-5d318188a5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:06.512233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:06.515398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 674/1365] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:06.822816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917cf78-80ec-422f-859d-432b97fdabb4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:06.823118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917cf78-80ec-422f-859d-432b97fdabb4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:06.823385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c917cf78-80ec-422f-859d-432b97fdabb4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:06.824866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 692/1366] 10.222.0.25 () {54 vars in 962 bytes} [Tue Sep 13 06:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:06.831257 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d69f7-31ba-4b10-9180-078c51c12f08 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:06.831555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d69f7-31ba-4b10-9180-078c51c12f08 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:06.831834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174d69f7-31ba-4b10-9180-078c51c12f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:06.914947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-174d69f7-31ba-4b10-9180-078c51c12f08 None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:06.916542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-174d69f7-31ba-4b10-9180-078c51c12f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:07.045588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 675/1367] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Sep 13 06:54:06 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:07.054359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917cf78-80ec-422f-859d-432b97fdabb4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:07.055880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c917cf78-80ec-422f-859d-432b97fdabb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:07.210895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c917cf78-80ec-422f-859d-432b97fdabb4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:07.232440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:07.332378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aec75d-bb26-4bd6-bbfc-d56aa1d585d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:07.333076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aec75d-bb26-4bd6-bbfc-d56aa1d585d8 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:07.333260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aec75d-bb26-4bd6-bbfc-d56aa1d585d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:07.333540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aec75d-bb26-4bd6-bbfc-d56aa1d585d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:07.335190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-52aec75d-bb26-4bd6-bbfc-d56aa1d585d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:07.431930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aec75d-bb26-4bd6-bbfc-d56aa1d585d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:07.433947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aec75d-bb26-4bd6-bbfc-d56aa1d585d8 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:07.456540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 693/1368] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:07.563328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c6392-de07-428a-bab2-66f54fd8177e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:07.563328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c6392-de07-428a-bab2-66f54fd8177e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:07.563328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258c6392-de07-428a-bab2-66f54fd8177e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:07.563328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 676/1369] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:07.565270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f0d1b-9fc0-43a0-aa12-e22e8771902b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:07.565561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f0d1b-9fc0-43a0-aa12-e22e8771902b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:07.565838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947f0d1b-9fc0-43a0-aa12-e22e8771902b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:07.601333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:07.653232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-947f0d1b-9fc0-43a0-aa12-e22e8771902b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:07.654780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-947f0d1b-9fc0-43a0-aa12-e22e8771902b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:07.779687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 694/1370] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:08.116099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb0049-b0a5-4020-819c-ea5036533f58 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:08.116099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb0049-b0a5-4020-819c-ea5036533f58 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:08.116099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdb0049-b0a5-4020-819c-ea5036533f58 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:08.118329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 677/1371] 10.222.0.25 () {54 vars in 964 bytes} [Tue Sep 13 06:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:08.122653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075dc4d-11f6-4d66-a8c8-009a26c770d0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:08.122653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075dc4d-11f6-4d66-a8c8-009a26c770d0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:08.124058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e075dc4d-11f6-4d66-a8c8-009a26c770d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:08.210448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e075dc4d-11f6-4d66-a8c8-009a26c770d0 None None] MFA Rules not processed for user `88520ce6493a44f39b19c67592f03632`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:08.211712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e075dc4d-11f6-4d66-a8c8-009a26c770d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:08.269653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:08.310802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 695/1372] 10.222.0.25 () {58 vars in 1061 bytes} [Tue Sep 13 06:54:08 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:08.327421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb0049-b0a5-4020-819c-ea5036533f58 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:08.331209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5bdb0049-b0a5-4020-819c-ea5036533f58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:08.399146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:08.474107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdb0049-b0a5-4020-819c-ea5036533f58 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:08.583654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf9db2c-5f41-4ec7-820e-5443d3c7d67f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:08.584293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9db2c-5f41-4ec7-820e-5443d3c7d67f service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:08.584539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9db2c-5f41-4ec7-820e-5443d3c7d67f service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:08.584807 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf9db2c-5f41-4ec7-820e-5443d3c7d67f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:08.586618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bcf9db2c-5f41-4ec7-820e-5443d3c7d67f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:08.748515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:08.768701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9db2c-5f41-4ec7-820e-5443d3c7d67f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:08.770204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf9db2c-5f41-4ec7-820e-5443d3c7d67f service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:08.795166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 678/1373] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:09.789460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f334-af16-4582-90d8-d3a0af6f198f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:09.790033 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f334-af16-4582-90d8-d3a0af6f198f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:09.790033 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9845f334-af16-4582-90d8-d3a0af6f198f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:09.791298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 696/1374] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:09.797094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c723ba9-883d-496e-89a3-9cbc5ac0f4a1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:09.797589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c723ba9-883d-496e-89a3-9cbc5ac0f4a1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:09.797589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c723ba9-883d-496e-89a3-9cbc5ac0f4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:09.867424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6c723ba9-883d-496e-89a3-9cbc5ac0f4a1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:09.868655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6c723ba9-883d-496e-89a3-9cbc5ac0f4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:09.970770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 679/1375] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:10.261781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845f334-af16-4582-90d8-d3a0af6f198f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:10.263749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9845f334-af16-4582-90d8-d3a0af6f198f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:10.437922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9845f334-af16-4582-90d8-d3a0af6f198f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:10.493276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:10.557648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2625464-3ad5-43dd-993f-23c86176ca4b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:10.558669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2625464-3ad5-43dd-993f-23c86176ca4b service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:10.558949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2625464-3ad5-43dd-993f-23c86176ca4b service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:10.559219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2625464-3ad5-43dd-993f-23c86176ca4b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:10.561208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f2625464-3ad5-43dd-993f-23c86176ca4b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:10.655771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2625464-3ad5-43dd-993f-23c86176ca4b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:10.657393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2625464-3ad5-43dd-993f-23c86176ca4b service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:10.676244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 697/1376] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:12.891781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa89156-c6d7-4dd8-b6c2-b61f622625df None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:12.892314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa89156-c6d7-4dd8-b6c2-b61f622625df None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:12.892314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa89156-c6d7-4dd8-b6c2-b61f622625df None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:12.893981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 680/1377] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:12.899355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939bad5-544b-46e3-9a31-a33aca4a1f7c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:12.899817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939bad5-544b-46e3-9a31-a33aca4a1f7c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:12.899897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7939bad5-544b-46e3-9a31-a33aca4a1f7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:12.969528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7939bad5-544b-46e3-9a31-a33aca4a1f7c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:12.970791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7939bad5-544b-46e3-9a31-a33aca4a1f7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:12.978487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:13.079230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 698/1378] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:13.363575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa89156-c6d7-4dd8-b6c2-b61f622625df None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:13.365137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5fa89156-c6d7-4dd8-b6c2-b61f622625df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:13.427694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:13.518844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa89156-c6d7-4dd8-b6c2-b61f622625df None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:13.644255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e655fca0-d202-48fb-a509-b83123f08823 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:13.644872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655fca0-d202-48fb-a509-b83123f08823 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:13.645146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655fca0-d202-48fb-a509-b83123f08823 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:13.645396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e655fca0-d202-48fb-a509-b83123f08823 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:13.647046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e655fca0-d202-48fb-a509-b83123f08823 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:13.666489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:13.741614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655fca0-d202-48fb-a509-b83123f08823 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:13.743230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e655fca0-d202-48fb-a509-b83123f08823 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:13.762014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 681/1379] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:15.964182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7db9ce-c7b1-4db2-afdf-6c86f4e42466 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:15.964756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7db9ce-c7b1-4db2-afdf-6c86f4e42466 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:15.964756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d7db9ce-c7b1-4db2-afdf-6c86f4e42466 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:15.970116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 699/1380] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:15.971765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780e865-7df1-4933-8b24-f2d766257a22 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:15.972221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780e865-7df1-4933-8b24-f2d766257a22 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:15.972314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780e865-7df1-4933-8b24-f2d766257a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:16.040445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f780e865-7df1-4933-8b24-f2d766257a22 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:16.041663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f780e865-7df1-4933-8b24-f2d766257a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:16.143003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:16.166895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 682/1381] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:16.451446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7db9ce-c7b1-4db2-afdf-6c86f4e42466 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:16.453499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9d7db9ce-c7b1-4db2-afdf-6c86f4e42466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:16.614810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:16.627142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d7db9ce-c7b1-4db2-afdf-6c86f4e42466 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:16.746747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497eb9e7-04d3-4bc8-ad63-3a2e35fbf659 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:16.747758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb9e7-04d3-4bc8-ad63-3a2e35fbf659 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:16.748106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb9e7-04d3-4bc8-ad63-3a2e35fbf659 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:16.748370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497eb9e7-04d3-4bc8-ad63-3a2e35fbf659 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:16.749997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-497eb9e7-04d3-4bc8-ad63-3a2e35fbf659 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:16.865574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eb9e7-04d3-4bc8-ad63-3a2e35fbf659 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:16.867419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497eb9e7-04d3-4bc8-ad63-3a2e35fbf659 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:16.879285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:16.889727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 700/1382] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:19.123971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493cdef-aa30-4812-98e3-6e9e52c3c84e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:19.124677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493cdef-aa30-4812-98e3-6e9e52c3c84e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:19.124677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8493cdef-aa30-4812-98e3-6e9e52c3c84e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:19.126107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 683/1383] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:19.132159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae51c8b-4684-41be-a707-ebcb4d3bd908 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:19.132608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae51c8b-4684-41be-a707-ebcb4d3bd908 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:19.132703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae51c8b-4684-41be-a707-ebcb4d3bd908 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:19.209753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cae51c8b-4684-41be-a707-ebcb4d3bd908 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:19.211362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cae51c8b-4684-41be-a707-ebcb4d3bd908 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:19.325333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 701/1384] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:19.614218 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493cdef-aa30-4812-98e3-6e9e52c3c84e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:19.616232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8493cdef-aa30-4812-98e3-6e9e52c3c84e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:19.781201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8493cdef-aa30-4812-98e3-6e9e52c3c84e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:19.868027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:19.921001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1335ee9a-474f-4c8c-b903-c7a7c493aa34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:19.921618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335ee9a-474f-4c8c-b903-c7a7c493aa34 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:19.921895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335ee9a-474f-4c8c-b903-c7a7c493aa34 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:19.922190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1335ee9a-474f-4c8c-b903-c7a7c493aa34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:19.923793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1335ee9a-474f-4c8c-b903-c7a7c493aa34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:20.023157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335ee9a-474f-4c8c-b903-c7a7c493aa34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:20.024771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1335ee9a-474f-4c8c-b903-c7a7c493aa34 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:20.043493 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 684/1385] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:22.382697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4731280-d3a6-42a3-aabc-21a3ceb7b39b None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:22.383351 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4731280-d3a6-42a3-aabc-21a3ceb7b39b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:22.383351 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4731280-d3a6-42a3-aabc-21a3ceb7b39b None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:22.384544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 702/1386] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:22.390355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a04c0c-e145-4a62-bcf3-c64d34841d2c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:22.390814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a04c0c-e145-4a62-bcf3-c64d34841d2c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:22.390920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a04c0c-e145-4a62-bcf3-c64d34841d2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:22.461883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b3a04c0c-e145-4a62-bcf3-c64d34841d2c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:22.463160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b3a04c0c-e145-4a62-bcf3-c64d34841d2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:22.499183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:22.570710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 685/1387] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:22.864781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4731280-d3a6-42a3-aabc-21a3ceb7b39b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:22.866149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f4731280-d3a6-42a3-aabc-21a3ceb7b39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:22.952405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:23.025149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4731280-d3a6-42a3-aabc-21a3ceb7b39b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:23.145950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a540bf03-2b0f-4206-b2e9-22cf2515aeb2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:23.146615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540bf03-2b0f-4206-b2e9-22cf2515aeb2 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:23.146892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540bf03-2b0f-4206-b2e9-22cf2515aeb2 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:23.147160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a540bf03-2b0f-4206-b2e9-22cf2515aeb2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:23.148805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a540bf03-2b0f-4206-b2e9-22cf2515aeb2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:23.189727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:23.243995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540bf03-2b0f-4206-b2e9-22cf2515aeb2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:23.245593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a540bf03-2b0f-4206-b2e9-22cf2515aeb2 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:23.264600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 703/1388] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:23.977901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e95ad2-2b46-441e-bf0c-19b2f8daf956 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:23.978530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e95ad2-2b46-441e-bf0c-19b2f8daf956 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:23.978530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e95ad2-2b46-441e-bf0c-19b2f8daf956 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:23.979801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 686/1389] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:23.985455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb83c4-b962-4409-a61a-56366ac0c9b6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:23.985906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb83c4-b962-4409-a61a-56366ac0c9b6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:23.986015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb83c4-b962-4409-a61a-56366ac0c9b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:24.055882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d9fb83c4-b962-4409-a61a-56366ac0c9b6 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:24.057281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9fb83c4-b962-4409-a61a-56366ac0c9b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:24.160213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 704/1390] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:24.460399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe09e4-e4a8-417f-83fb-8614766534b7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:24.461018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe09e4-e4a8-417f-83fb-8614766534b7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:24.461517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fe09e4-e4a8-417f-83fb-8614766534b7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:24.463579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 687/1391] 10.222.0.25 () {54 vars in 966 bytes} [Tue Sep 13 06:54:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:24.470271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f64f1b-77cb-44a5-bfcb-70d6faebb666 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:24.470888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f64f1b-77cb-44a5-bfcb-70d6faebb666 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:24.471258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f64f1b-77cb-44a5-bfcb-70d6faebb666 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:24.487766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:24.557159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-42f64f1b-77cb-44a5-bfcb-70d6faebb666 None None] MFA Rules not processed for user `3505c8965c4a42e493a2f5916cab56dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:24.558935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-42f64f1b-77cb-44a5-bfcb-70d6faebb666 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:24.647133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 705/1392] 10.222.0.25 () {58 vars in 1063 bytes} [Tue Sep 13 06:54:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:24.660270 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe09e4-e4a8-417f-83fb-8614766534b7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:24.661823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-04fe09e4-e4a8-417f-83fb-8614766534b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:24.768785 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fe09e4-e4a8-417f-83fb-8614766534b7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:24.829804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:24.847142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6e62c9-4577-46ba-b0be-60017eea1939 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3505c8965c4a42e493a2f5916cab56dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:24.847840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e62c9-4577-46ba-b0be-60017eea1939 service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:24.848122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e62c9-4577-46ba-b0be-60017eea1939 service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:24.848397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6e62c9-4577-46ba-b0be-60017eea1939 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:24.850007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e6e62c9-4577-46ba-b0be-60017eea1939 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:24.967176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e62c9-4577-46ba-b0be-60017eea1939 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:24.968763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6e62c9-4577-46ba-b0be-60017eea1939 service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:24.987080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 688/1393] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:25.461448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2320a-b791-43d2-a76c-c15b9e5a7e8e None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:25.462060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2320a-b791-43d2-a76c-c15b9e5a7e8e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:25.462060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b2320a-b791-43d2-a76c-c15b9e5a7e8e None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:25.463422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 706/1394] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:25.469201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c12e7-40dd-49b5-a9bd-be451f60a5ac None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:25.469507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c12e7-40dd-49b5-a9bd-be451f60a5ac None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:25.469832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20c12e7-40dd-49b5-a9bd-be451f60a5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:25.545222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f20c12e7-40dd-49b5-a9bd-be451f60a5ac None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:25.546698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f20c12e7-40dd-49b5-a9bd-be451f60a5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:25.597733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:25.664620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 689/1395] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 06:54:25.756113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840147bf-5b1f-43c4-bbe3-411997ecfdcf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:25.756399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840147bf-5b1f-43c4-bbe3-411997ecfdcf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:25.756670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840147bf-5b1f-43c4-bbe3-411997ecfdcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:25.824377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-840147bf-5b1f-43c4-bbe3-411997ecfdcf None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:25.825660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-840147bf-5b1f-43c4-bbe3-411997ecfdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:25.885271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:25.917537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 707/1396] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 06:54:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:25.964937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53971951-61d3-4130-9a81-ffc55d5da3bd None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:25.965306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53971951-61d3-4130-9a81-ffc55d5da3bd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:25.966031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53971951-61d3-4130-9a81-ffc55d5da3bd None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:25.966924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 690/1397] 10.222.0.25 () {54 vars in 943 bytes} [Tue Sep 13 06:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:25.969469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840147bf-5b1f-43c4-bbe3-411997ecfdcf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:25.970625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-840147bf-5b1f-43c4-bbe3-411997ecfdcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:25.974454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efde90-6090-4a4a-a040-1a04f308d55e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:25.975653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efde90-6090-4a4a-a040-1a04f308d55e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:25.975653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9efde90-6090-4a4a-a040-1a04f308d55e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:26.069708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f9efde90-6090-4a4a-a040-1a04f308d55e None None] MFA Rules not processed for user `4815ef133a3a4604bc467c222d450fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:26.070953 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f9efde90-6090-4a4a-a040-1a04f308d55e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:26.096575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840147bf-5b1f-43c4-bbe3-411997ecfdcf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:26.178796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:26.189184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 691/1398] 10.222.0.25 () {58 vars in 1040 bytes} [Tue Sep 13 06:54:25 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:26.198917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efde90-6090-4a4a-a040-1a04f308d55e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:26.203820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:26.209248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9efde90-6090-4a4a-a040-1a04f308d55e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:26.215847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d334b619-3611-44d2-9593-1ec66840d507 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:26.216473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334b619-3611-44d2-9593-1ec66840d507 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:26.216753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334b619-3611-44d2-9593-1ec66840d507 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:26.217028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d334b619-3611-44d2-9593-1ec66840d507 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:26.218669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d334b619-3611-44d2-9593-1ec66840d507 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:26.316570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334b619-3611-44d2-9593-1ec66840d507 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:26.318370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d334b619-3611-44d2-9593-1ec66840d507 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:26.320931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5297cc-5342-4f9f-a4a2-a9ea8f26d23a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:26.321551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5297cc-5342-4f9f-a4a2-a9ea8f26d23a service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:26.321829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5297cc-5342-4f9f-a4a2-a9ea8f26d23a service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:26.322105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5297cc-5342-4f9f-a4a2-a9ea8f26d23a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:26.323809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-da5297cc-5342-4f9f-a4a2-a9ea8f26d23a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:26.337098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 708/1399] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:26.415083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5297cc-5342-4f9f-a4a2-a9ea8f26d23a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:26.416868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5297cc-5342-4f9f-a4a2-a9ea8f26d23a service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:26.426632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 692/1400] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 06:54:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 06:54:28.549391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d3da-23b7-40d6-a1b5-46580f938987 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:28.549997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d3da-23b7-40d6-a1b5-46580f938987 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:28.549997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973d3da-23b7-40d6-a1b5-46580f938987 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:28.554401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 709/1401] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:28.557063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d5062-df0d-4b87-8472-acb055171b7d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:28.557510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d5062-df0d-4b87-8472-acb055171b7d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:28.558502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d5062-df0d-4b87-8472-acb055171b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:28.603519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:28.642825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9d0d5062-df0d-4b87-8472-acb055171b7d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:28.644163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9d0d5062-df0d-4b87-8472-acb055171b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:28.774487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 693/1402] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:29.071214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d3da-23b7-40d6-a1b5-46580f938987 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:29.072755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9973d3da-23b7-40d6-a1b5-46580f938987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:29.171049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:29.233724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9973d3da-23b7-40d6-a1b5-46580f938987 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:29.363857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0278b53f-59c0-4511-b32c-9867a5b6cbaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:29.364471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278b53f-59c0-4511-b32c-9867a5b6cbaa service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:29.364744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278b53f-59c0-4511-b32c-9867a5b6cbaa service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:29.365011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0278b53f-59c0-4511-b32c-9867a5b6cbaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:29.366674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0278b53f-59c0-4511-b32c-9867a5b6cbaa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:29.432045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:29.470407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278b53f-59c0-4511-b32c-9867a5b6cbaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:29.472076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0278b53f-59c0-4511-b32c-9867a5b6cbaa service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:29.497303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 710/1403] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:31.740013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b99fd5-d956-4820-9069-0abdf7930167 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:31.740575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b99fd5-d956-4820-9069-0abdf7930167 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:31.740575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b99fd5-d956-4820-9069-0abdf7930167 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:31.745760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 694/1404] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:31.747234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259c0d9-ea18-4dfd-920d-2a2f27f1ab0e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:31.747704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259c0d9-ea18-4dfd-920d-2a2f27f1ab0e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:31.747804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6259c0d9-ea18-4dfd-920d-2a2f27f1ab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:31.825388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6259c0d9-ea18-4dfd-920d-2a2f27f1ab0e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:31.827092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6259c0d9-ea18-4dfd-920d-2a2f27f1ab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:31.933506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 711/1405] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 06:54:32.222843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b99fd5-d956-4820-9069-0abdf7930167 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:32.224400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-17b99fd5-d956-4820-9069-0abdf7930167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:32.348111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:32.398891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b99fd5-d956-4820-9069-0abdf7930167 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:32.532106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0d996c-fd69-4cf7-9eab-255bca89b5f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:32.532631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d996c-fd69-4cf7-9eab-255bca89b5f0 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:32.532908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d996c-fd69-4cf7-9eab-255bca89b5f0 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:32.533167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0d996c-fd69-4cf7-9eab-255bca89b5f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:32.536137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7c0d996c-fd69-4cf7-9eab-255bca89b5f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:32.622279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:32.648228 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d996c-fd69-4cf7-9eab-255bca89b5f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:32.649824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0d996c-fd69-4cf7-9eab-255bca89b5f0 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:32.668698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 695/1406] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:34.865803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af7752-1aea-4928-965c-dc7cc2b2bc1c None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:34.866400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af7752-1aea-4928-965c-dc7cc2b2bc1c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:34.866400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af7752-1aea-4928-965c-dc7cc2b2bc1c None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:34.872398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 712/1407] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:34.874973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b77210-b70b-4073-bf36-1bd1bc47da41 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:34.875853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b77210-b70b-4073-bf36-1bd1bc47da41 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:34.876196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b77210-b70b-4073-bf36-1bd1bc47da41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:34.956902 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-91b77210-b70b-4073-bf36-1bd1bc47da41 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:34.958216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-91b77210-b70b-4073-bf36-1bd1bc47da41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:35.073440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 696/1408] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:54:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:35.362935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af7752-1aea-4928-965c-dc7cc2b2bc1c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:35.364475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-25af7752-1aea-4928-965c-dc7cc2b2bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:35.406215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:35.536894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af7752-1aea-4928-965c-dc7cc2b2bc1c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:35.664982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:35.672936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d260250-2f52-4a07-8fd2-1f7e03fd0ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:35.673643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d260250-2f52-4a07-8fd2-1f7e03fd0ae8 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:35.673989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d260250-2f52-4a07-8fd2-1f7e03fd0ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:35.674296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d260250-2f52-4a07-8fd2-1f7e03fd0ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:35.675980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5d260250-2f52-4a07-8fd2-1f7e03fd0ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:35.779686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d260250-2f52-4a07-8fd2-1f7e03fd0ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:35.781371 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d260250-2f52-4a07-8fd2-1f7e03fd0ae8 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:35.799964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 713/1409] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:45.065411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=79 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:45.797841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=48 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:49.114824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b6a85-5998-4a1b-a618-1d3e4c856d18 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:49.115410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b6a85-5998-4a1b-a618-1d3e4c856d18 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:49.115410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0b6a85-5998-4a1b-a618-1d3e4c856d18 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:49.116749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 697/1410] 10.222.0.25 () {54 vars in 918 bytes} [Tue Sep 13 06:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:49.122308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c857c7e-3882-4fad-a870-53661d30b296 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:49.122769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c857c7e-3882-4fad-a870-53661d30b296 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:49.122875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c857c7e-3882-4fad-a870-53661d30b296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:49.207832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9c857c7e-3882-4fad-a870-53661d30b296 None None] MFA Rules not processed for user `e8202ad6007246078c4d25365dc5a0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:49.209072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9c857c7e-3882-4fad-a870-53661d30b296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:49.325113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 714/1411] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Sep 13 06:54:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:49.340124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1c8f3-e785-4335-a57b-0b830f859978 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:49.340410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1c8f3-e785-4335-a57b-0b830f859978 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:49.340682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1c8f3-e785-4335-a57b-0b830f859978 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:49.342036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 698/1412] 10.222.0.25 () {54 vars in 918 bytes} [Tue Sep 13 06:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:49.347142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4e045-2357-441e-8be7-65e1ae0ec5ce None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:49.347429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4e045-2357-441e-8be7-65e1ae0ec5ce None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:49.347702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4e045-2357-441e-8be7-65e1ae0ec5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:49.425650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:49.429004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-86c4e045-2357-441e-8be7-65e1ae0ec5ce None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:49.430865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-86c4e045-2357-441e-8be7-65e1ae0ec5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:49.559242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 715/1413] 10.222.0.25 () {58 vars in 1015 bytes} [Tue Sep 13 06:54:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:52.132772 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c9afd-ade1-41cb-9c04-289e8cee1b6e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:52.133373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c9afd-ade1-41cb-9c04-289e8cee1b6e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:52.133373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018c9afd-ade1-41cb-9c04-289e8cee1b6e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:52.135456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 699/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:54:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:52.143684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1a640-9ba3-4dc6-ba30-8a0c4b45dddb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:52.144174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1a640-9ba3-4dc6-ba30-8a0c4b45dddb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:52.144174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d1a640-9ba3-4dc6-ba30-8a0c4b45dddb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:52.214887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-58d1a640-9ba3-4dc6-ba30-8a0c4b45dddb None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:52.216261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-58d1a640-9ba3-4dc6-ba30-8a0c4b45dddb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:52.271962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:52.315897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 716/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:54:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:52.604119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c9afd-ade1-41cb-9c04-289e8cee1b6e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:52.607273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-018c9afd-ade1-41cb-9c04-289e8cee1b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:52.766581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018c9afd-ade1-41cb-9c04-289e8cee1b6e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:52.882755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:53.044324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94255a76-a666-471f-9f8b-02bcc156fa3f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:53.044938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94255a76-a666-471f-9f8b-02bcc156fa3f service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:53.045242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94255a76-a666-471f-9f8b-02bcc156fa3f service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:53.045461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94255a76-a666-471f-9f8b-02bcc156fa3f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:53.047141 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-94255a76-a666-471f-9f8b-02bcc156fa3f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:53.158104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94255a76-a666-471f-9f8b-02bcc156fa3f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:53.159676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94255a76-a666-471f-9f8b-02bcc156fa3f service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:53.180698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 700/1416] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 578 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:54.318657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a23d2-1fc1-4855-993a-7acdb76dc5a3 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:54.319293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a23d2-1fc1-4855-993a-7acdb76dc5a3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:54.319293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a23d2-1fc1-4855-993a-7acdb76dc5a3 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:54.320546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 717/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:54.328209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320dcc9-08bf-4e13-840a-64c86195adce None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:54.328992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320dcc9-08bf-4e13-840a-64c86195adce None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:54.329178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4320dcc9-08bf-4e13-840a-64c86195adce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:54.412503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4320dcc9-08bf-4e13-840a-64c86195adce None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:54.414323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4320dcc9-08bf-4e13-840a-64c86195adce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:54.450388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:54.524228 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 701/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:54:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:54.678027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a23d2-1fc1-4855-993a-7acdb76dc5a3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:54.680307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d36a23d2-1fc1-4855-993a-7acdb76dc5a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:54.838131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a23d2-1fc1-4855-993a-7acdb76dc5a3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:54.881350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:54.965603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abee8d0a-b947-4da7-a1e4-b5c0165f9b84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:54.966254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee8d0a-b947-4da7-a1e4-b5c0165f9b84 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:54.966536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee8d0a-b947-4da7-a1e4-b5c0165f9b84 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:54.966808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abee8d0a-b947-4da7-a1e4-b5c0165f9b84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:54.968431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-abee8d0a-b947-4da7-a1e4-b5c0165f9b84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:55.064870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee8d0a-b947-4da7-a1e4-b5c0165f9b84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:55.066510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abee8d0a-b947-4da7-a1e4-b5c0165f9b84 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:55.084981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 718/1419] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:56.232733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae29d99-0989-4651-ba9e-2aed1dab13b6 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:56.233397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae29d99-0989-4651-ba9e-2aed1dab13b6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:56.233397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae29d99-0989-4651-ba9e-2aed1dab13b6 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:56.234694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 702/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:54:56.242148 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edc31a-f062-48d2-9b4e-94b5f357028b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:56.242648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edc31a-f062-48d2-9b4e-94b5f357028b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:56.242765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1edc31a-f062-48d2-9b4e-94b5f357028b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:56.309672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:56.332599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a1edc31a-f062-48d2-9b4e-94b5f357028b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:56.333815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a1edc31a-f062-48d2-9b4e-94b5f357028b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:56.437764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 719/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:54:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:56.592717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae29d99-0989-4651-ba9e-2aed1dab13b6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:56.594324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dae29d99-0989-4651-ba9e-2aed1dab13b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:56.759858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae29d99-0989-4651-ba9e-2aed1dab13b6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:56.773338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:56.892376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b28186-73c0-406e-90cc-c5c79d1d7d60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:56.892901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b28186-73c0-406e-90cc-c5c79d1d7d60 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:56.893170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b28186-73c0-406e-90cc-c5c79d1d7d60 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:56.893430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b28186-73c0-406e-90cc-c5c79d1d7d60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:56.895047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a8b28186-73c0-406e-90cc-c5c79d1d7d60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:56.989160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b28186-73c0-406e-90cc-c5c79d1d7d60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:56.990803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b28186-73c0-406e-90cc-c5c79d1d7d60 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:57.008817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 703/1422] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:54:58.146526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450fa37-6558-46a6-a275-aaea34e725d5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:58.147134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450fa37-6558-46a6-a275-aaea34e725d5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:58.147134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7450fa37-6558-46a6-a275-aaea34e725d5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:58.148449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 720/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:54:58.156965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9c8ed-c959-40e1-be93-f4ac030adb3c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:58.157463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9c8ed-c959-40e1-be93-f4ac030adb3c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:58.157561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be9c8ed-c959-40e1-be93-f4ac030adb3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:58.187707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:58.246313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0be9c8ed-c959-40e1-be93-f4ac030adb3c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:54:58.247466 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0be9c8ed-c959-40e1-be93-f4ac030adb3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:58.351636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 704/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:54:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:54:58.505318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450fa37-6558-46a6-a275-aaea34e725d5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:54:58.507786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7450fa37-6558-46a6-a275-aaea34e725d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:58.629565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:58.667023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7450fa37-6558-46a6-a275-aaea34e725d5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:54:58.796985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72e53a-377c-4824-baf2-3d59a0bdeca5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:54:58.797514 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72e53a-377c-4824-baf2-3d59a0bdeca5 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:54:58.797795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72e53a-377c-4824-baf2-3d59a0bdeca5 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:54:58.798060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72e53a-377c-4824-baf2-3d59a0bdeca5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:54:58.799707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dc72e53a-377c-4824-baf2-3d59a0bdeca5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:54:58.893255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:54:58.912513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72e53a-377c-4824-baf2-3d59a0bdeca5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:54:58.914157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72e53a-377c-4824-baf2-3d59a0bdeca5 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:54:58.933729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 721/1425] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:54:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:00.071453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34e67-77bc-4cff-a820-7692baeb04a7 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:00.072042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34e67-77bc-4cff-a820-7692baeb04a7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:00.072042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a34e67-77bc-4cff-a820-7692baeb04a7 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:00.073261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 705/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:00.080669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336eb4e-758b-47d1-ba16-42ace899b432 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:00.081160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336eb4e-758b-47d1-ba16-42ace899b432 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:00.081270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336eb4e-758b-47d1-ba16-42ace899b432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:00.160528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7336eb4e-758b-47d1-ba16-42ace899b432 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:00.161730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7336eb4e-758b-47d1-ba16-42ace899b432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:00.262715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 722/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:55:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:00.412150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34e67-77bc-4cff-a820-7692baeb04a7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:00.413671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-32a34e67-77bc-4cff-a820-7692baeb04a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:00.505884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:00.579646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a34e67-77bc-4cff-a820-7692baeb04a7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:00.708272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d05f32-bdce-4e5d-9478-84177377e3ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:00.709079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d05f32-bdce-4e5d-9478-84177377e3ed service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:00.709549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d05f32-bdce-4e5d-9478-84177377e3ed service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:00.710013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d05f32-bdce-4e5d-9478-84177377e3ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:00.712796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-93d05f32-bdce-4e5d-9478-84177377e3ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:00.780306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:00.830687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d05f32-bdce-4e5d-9478-84177377e3ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:00.832344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d05f32-bdce-4e5d-9478-84177377e3ed service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:00.852089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 706/1428] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:01.925973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6add4-23b6-45c1-9034-0078734dca44 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:01.926637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6add4-23b6-45c1-9034-0078734dca44 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:01.926637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc6add4-23b6-45c1-9034-0078734dca44 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:01.927920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 723/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:01.935571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccad168-a124-4928-b482-239b08291fd9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:01.936051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccad168-a124-4928-b482-239b08291fd9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:01.936162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eccad168-a124-4928-b482-239b08291fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:02.014119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eccad168-a124-4928-b482-239b08291fd9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:02.015334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eccad168-a124-4928-b482-239b08291fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:02.116607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 707/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:55:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:02.269039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6add4-23b6-45c1-9034-0078734dca44 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:02.270601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ddc6add4-23b6-45c1-9034-0078734dca44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:02.324762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:02.440442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc6add4-23b6-45c1-9034-0078734dca44 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:02.562773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4569780a-23ea-478f-b1df-906ff136dbb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:02.563297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569780a-23ea-478f-b1df-906ff136dbb5 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:02.563577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569780a-23ea-478f-b1df-906ff136dbb5 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:02.563849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4569780a-23ea-478f-b1df-906ff136dbb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:02.565470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4569780a-23ea-478f-b1df-906ff136dbb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:02.572530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:02.661668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569780a-23ea-478f-b1df-906ff136dbb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:02.663334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4569780a-23ea-478f-b1df-906ff136dbb5 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:02.684246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 724/1431] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:03.749226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9992f55-9b02-45f6-b8c6-edcd9b88a1b0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:03.749813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9992f55-9b02-45f6-b8c6-edcd9b88a1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:03.749813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9992f55-9b02-45f6-b8c6-edcd9b88a1b0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:03.751166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 708/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:03.758757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9626c2-5732-437d-9dca-d397b02ded4f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:03.759537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9626c2-5732-437d-9dca-d397b02ded4f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:03.759681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9626c2-5732-437d-9dca-d397b02ded4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:03.841353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0e9626c2-5732-437d-9dca-d397b02ded4f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:03.842633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0e9626c2-5732-437d-9dca-d397b02ded4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:03.921850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:03.952442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 725/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:55:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:04.108981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9992f55-9b02-45f6-b8c6-edcd9b88a1b0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:04.111519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c9992f55-9b02-45f6-b8c6-edcd9b88a1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:04.134581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:04.278653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9992f55-9b02-45f6-b8c6-edcd9b88a1b0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:04.374957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:04.402690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695961be-ca52-40f4-9b25-3a1ab4a65ff1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:04.403314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695961be-ca52-40f4-9b25-3a1ab4a65ff1 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:04.403620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695961be-ca52-40f4-9b25-3a1ab4a65ff1 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:04.403893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695961be-ca52-40f4-9b25-3a1ab4a65ff1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:04.405510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-695961be-ca52-40f4-9b25-3a1ab4a65ff1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:04.500580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695961be-ca52-40f4-9b25-3a1ab4a65ff1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:04.502220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695961be-ca52-40f4-9b25-3a1ab4a65ff1 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:04.521545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 709/1434] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:05.589795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e8cf3-9a13-4cb0-b29a-a43cb4446090 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:05.590430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e8cf3-9a13-4cb0-b29a-a43cb4446090 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:05.590430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421e8cf3-9a13-4cb0-b29a-a43cb4446090 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:05.591837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 726/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:05.599463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c886cbf-dc9e-4ef0-a889-7af5ad72ad46 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:05.600017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c886cbf-dc9e-4ef0-a889-7af5ad72ad46 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:05.600017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c886cbf-dc9e-4ef0-a889-7af5ad72ad46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:05.683180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6c886cbf-dc9e-4ef0-a889-7af5ad72ad46 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:05.684399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6c886cbf-dc9e-4ef0-a889-7af5ad72ad46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:05.730711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:05.789402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 710/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:55:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:05.943692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e8cf3-9a13-4cb0-b29a-a43cb4446090 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:05.945229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-421e8cf3-9a13-4cb0-b29a-a43cb4446090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:06.120193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421e8cf3-9a13-4cb0-b29a-a43cb4446090 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:06.179435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:06.253764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc284e-1757-4f05-8d76-16fdf759088d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:06.254419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc284e-1757-4f05-8d76-16fdf759088d service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:06.254706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc284e-1757-4f05-8d76-16fdf759088d service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:06.254971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc284e-1757-4f05-8d76-16fdf759088d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:06.256591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-72dc284e-1757-4f05-8d76-16fdf759088d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:06.352690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc284e-1757-4f05-8d76-16fdf759088d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:06.354345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dc284e-1757-4f05-8d76-16fdf759088d service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:06.375980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 727/1437] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:07.425125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5e03f-1fa8-4397-b9aa-133589b72de3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:07.426045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5e03f-1fa8-4397-b9aa-133589b72de3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:07.426045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b5e03f-1fa8-4397-b9aa-133589b72de3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:07.428127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 711/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:55:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:07.439001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b20180-1347-42c8-8cd7-b7e49da575d7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:07.439822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b20180-1347-42c8-8cd7-b7e49da575d7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:07.439822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b20180-1347-42c8-8cd7-b7e49da575d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:07.520987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-91b20180-1347-42c8-8cd7-b7e49da575d7 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:07.522212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-91b20180-1347-42c8-8cd7-b7e49da575d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:07.524263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:07.629706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 728/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:55:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:07.783019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5e03f-1fa8-4397-b9aa-133589b72de3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:07.785292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a2b5e03f-1fa8-4397-b9aa-133589b72de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:08.023434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b5e03f-1fa8-4397-b9aa-133589b72de3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:08.067994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:08.189837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3659201c-72d5-47c6-a0ae-71bcc2c1f94a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:08.190497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659201c-72d5-47c6-a0ae-71bcc2c1f94a service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:08.190773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659201c-72d5-47c6-a0ae-71bcc2c1f94a service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:08.191053 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3659201c-72d5-47c6-a0ae-71bcc2c1f94a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:08.192694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3659201c-72d5-47c6-a0ae-71bcc2c1f94a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:08.323734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659201c-72d5-47c6-a0ae-71bcc2c1f94a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:08.326319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3659201c-72d5-47c6-a0ae-71bcc2c1f94a service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:08.352978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 712/1440] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 571 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:10.476004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e5f8e-ed8f-4f19-9f4e-2815458160dc None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:10.476583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e5f8e-ed8f-4f19-9f4e-2815458160dc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:10.476716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365e5f8e-ed8f-4f19-9f4e-2815458160dc None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:10.478777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 729/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:10.484423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e204f8-fe2f-4fbd-a740-efe3f3f5acc1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:10.484875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e204f8-fe2f-4fbd-a740-efe3f3f5acc1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:10.484987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e204f8-fe2f-4fbd-a740-efe3f3f5acc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:10.536877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:10.564854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-37e204f8-fe2f-4fbd-a740-efe3f3f5acc1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:10.566127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-37e204f8-fe2f-4fbd-a740-efe3f3f5acc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:10.672456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 713/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:10.969394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e5f8e-ed8f-4f19-9f4e-2815458160dc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:10.971916 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-365e5f8e-ed8f-4f19-9f4e-2815458160dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:11.097235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365e5f8e-ed8f-4f19-9f4e-2815458160dc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:11.130417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:11.175196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c773012-ba03-410f-8442-8a375de7ce78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:11.175820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c773012-ba03-410f-8442-8a375de7ce78 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:11.176100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c773012-ba03-410f-8442-8a375de7ce78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:11.176373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c773012-ba03-410f-8442-8a375de7ce78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:11.177981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2c773012-ba03-410f-8442-8a375de7ce78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:11.293252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c773012-ba03-410f-8442-8a375de7ce78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:11.294699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c773012-ba03-410f-8442-8a375de7ce78 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:11.313808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 730/1443] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:13.675335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac9c53-3688-463b-97bd-14bbf84e0971 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:13.675924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac9c53-3688-463b-97bd-14bbf84e0971 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:13.675924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ac9c53-3688-463b-97bd-14bbf84e0971 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:13.677233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 714/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:13.684195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dac9b-1db4-4b64-98a7-e30815b23843 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:13.685017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dac9b-1db4-4b64-98a7-e30815b23843 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:13.685270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dac9b-1db4-4b64-98a7-e30815b23843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:13.754403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-762dac9b-1db4-4b64-98a7-e30815b23843 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:13.755632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-762dac9b-1db4-4b64-98a7-e30815b23843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:13.768874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:13.871075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 731/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:14.154942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac9c53-3688-463b-97bd-14bbf84e0971 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:14.156481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c6ac9c53-3688-463b-97bd-14bbf84e0971 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:14.263235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ac9c53-3688-463b-97bd-14bbf84e0971 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:14.276457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:14.347264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277df7ba-a9d2-4565-8d16-0f8c5a6b0bb8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:14.347895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277df7ba-a9d2-4565-8d16-0f8c5a6b0bb8 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:14.348177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277df7ba-a9d2-4565-8d16-0f8c5a6b0bb8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:14.348451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277df7ba-a9d2-4565-8d16-0f8c5a6b0bb8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:14.350176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-277df7ba-a9d2-4565-8d16-0f8c5a6b0bb8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:14.464998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277df7ba-a9d2-4565-8d16-0f8c5a6b0bb8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:14.466527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277df7ba-a9d2-4565-8d16-0f8c5a6b0bb8 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:14.484702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 715/1446] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:16.946185 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16138854-2c79-4614-9427-5588259dba9f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:16.946766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16138854-2c79-4614-9427-5588259dba9f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:16.946766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16138854-2c79-4614-9427-5588259dba9f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:16.948011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 732/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:16.954534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0295d5f-33c5-4597-a80b-003090e1ee96 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:16.955190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0295d5f-33c5-4597-a80b-003090e1ee96 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:16.955284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0295d5f-33c5-4597-a80b-003090e1ee96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:17.014845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:17.038040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d0295d5f-33c5-4597-a80b-003090e1ee96 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:17.039942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d0295d5f-33c5-4597-a80b-003090e1ee96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:17.147551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 716/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:17.438510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16138854-2c79-4614-9427-5588259dba9f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:17.440075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-16138854-2c79-4614-9427-5588259dba9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:17.547666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16138854-2c79-4614-9427-5588259dba9f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:17.589186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:17.624342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534b30e9-08f8-4b04-bbd1-ad7c339dd8c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:17.624951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b30e9-08f8-4b04-bbd1-ad7c339dd8c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:17.625229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b30e9-08f8-4b04-bbd1-ad7c339dd8c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:17.625491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534b30e9-08f8-4b04-bbd1-ad7c339dd8c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:17.627101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-534b30e9-08f8-4b04-bbd1-ad7c339dd8c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:17.744529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b30e9-08f8-4b04-bbd1-ad7c339dd8c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:17.745931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534b30e9-08f8-4b04-bbd1-ad7c339dd8c2 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:17.764429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 733/1449] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:20.072127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512b8d0-eb93-487f-b560-10e156671bf0 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:20.072708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512b8d0-eb93-487f-b560-10e156671bf0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:20.072708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3512b8d0-eb93-487f-b560-10e156671bf0 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:20.073947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 717/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 06:55:20.079799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1d55a-022a-48f9-9e8a-f1a885743bd4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:20.080283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1d55a-022a-48f9-9e8a-f1a885743bd4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:20.080423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1d55a-022a-48f9-9e8a-f1a885743bd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:20.147114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b2a1d55a-022a-48f9-9e8a-f1a885743bd4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:20.148358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b2a1d55a-022a-48f9-9e8a-f1a885743bd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:20.169937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:20.248984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 734/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:20.532019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512b8d0-eb93-487f-b560-10e156671bf0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:20.534234 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3512b8d0-eb93-487f-b560-10e156671bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:20.655670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3512b8d0-eb93-487f-b560-10e156671bf0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:20.677295 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:20.737763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e42eec-411b-4e57-9652-1676bb03d7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:20.738408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e42eec-411b-4e57-9652-1676bb03d7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:20.738684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e42eec-411b-4e57-9652-1676bb03d7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:20.738922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e42eec-411b-4e57-9652-1676bb03d7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:20.740522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c0e42eec-411b-4e57-9652-1676bb03d7a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:20.862816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e42eec-411b-4e57-9652-1676bb03d7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:20.864284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e42eec-411b-4e57-9652-1676bb03d7a9 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:20.882877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 718/1452] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:23.093467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e428a59-7b69-466d-91e0-4b1437fef3dd None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:23.094446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e428a59-7b69-466d-91e0-4b1437fef3dd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:23.094446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e428a59-7b69-466d-91e0-4b1437fef3dd None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:23.096966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 735/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:23 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:23.106512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b60d7f-1b85-4115-a350-8cc143ced2f8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:23.107025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b60d7f-1b85-4115-a350-8cc143ced2f8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:23.107232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b60d7f-1b85-4115-a350-8cc143ced2f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:23.181959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:23.185833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a0b60d7f-1b85-4115-a350-8cc143ced2f8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:23.187523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a0b60d7f-1b85-4115-a350-8cc143ced2f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:23.287873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 719/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:23.576030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e428a59-7b69-466d-91e0-4b1437fef3dd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:23.577655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8e428a59-7b69-466d-91e0-4b1437fef3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:23.690454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e428a59-7b69-466d-91e0-4b1437fef3dd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:23.741795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:23.767719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d468ed25-8003-41e1-8468-f9be60f319cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:23.768332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468ed25-8003-41e1-8468-f9be60f319cc service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:23.768605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468ed25-8003-41e1-8468-f9be60f319cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:23.768880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d468ed25-8003-41e1-8468-f9be60f319cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:23.770539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d468ed25-8003-41e1-8468-f9be60f319cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:23.887089 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468ed25-8003-41e1-8468-f9be60f319cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:23.888701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d468ed25-8003-41e1-8468-f9be60f319cc service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:23.908798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 736/1455] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:26.150704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b5594-595b-4eba-bf82-dfe68e62d4ee None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:26.151727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b5594-595b-4eba-bf82-dfe68e62d4ee None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:26.151727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58b5594-595b-4eba-bf82-dfe68e62d4ee None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:26.154071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 720/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:26.160231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6fd1-58b6-4409-aecc-40c9ac6d82a1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:26.160708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6fd1-58b6-4409-aecc-40c9ac6d82a1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:26.160812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808b6fd1-58b6-4409-aecc-40c9ac6d82a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:26.230279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-808b6fd1-58b6-4409-aecc-40c9ac6d82a1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:26.231510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-808b6fd1-58b6-4409-aecc-40c9ac6d82a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:26.265535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:26.338814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 737/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:26.623183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b5594-595b-4eba-bf82-dfe68e62d4ee None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:26.624849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f58b5594-595b-4eba-bf82-dfe68e62d4ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:26.737816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58b5594-595b-4eba-bf82-dfe68e62d4ee None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:26.769229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:26.822872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db67e786-9676-4cde-b196-7045ea12ee07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:26.823486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67e786-9676-4cde-b196-7045ea12ee07 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:26.823762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67e786-9676-4cde-b196-7045ea12ee07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:26.824029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db67e786-9676-4cde-b196-7045ea12ee07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:26.825874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-db67e786-9676-4cde-b196-7045ea12ee07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:26.941960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67e786-9676-4cde-b196-7045ea12ee07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:26.943360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db67e786-9676-4cde-b196-7045ea12ee07 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:26.961848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 721/1458] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:29.159729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22a878-cf1d-4c8b-a37b-afba1c625ebe None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:29.160275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22a878-cf1d-4c8b-a37b-afba1c625ebe None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:29.160275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22a878-cf1d-4c8b-a37b-afba1c625ebe None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:29.161650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 738/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:29.168101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e796d-bc82-4f68-85f8-8dda30a40a68 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:29.168565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e796d-bc82-4f68-85f8-8dda30a40a68 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:29.168675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34e796d-bc82-4f68-85f8-8dda30a40a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:29.246143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b34e796d-bc82-4f68-85f8-8dda30a40a68 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:29.248105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b34e796d-bc82-4f68-85f8-8dda30a40a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:29.257453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:29.349254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 722/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:29.639101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22a878-cf1d-4c8b-a37b-afba1c625ebe None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:29.640670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bb22a878-cf1d-4c8b-a37b-afba1c625ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:29.751279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22a878-cf1d-4c8b-a37b-afba1c625ebe None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:29.820265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:29.838695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a66860-7661-4f59-b3ef-93e7d4fd7126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:29.839315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a66860-7661-4f59-b3ef-93e7d4fd7126 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:29.839596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a66860-7661-4f59-b3ef-93e7d4fd7126 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:29.839864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a66860-7661-4f59-b3ef-93e7d4fd7126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:29.841546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-14a66860-7661-4f59-b3ef-93e7d4fd7126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:29.955460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a66860-7661-4f59-b3ef-93e7d4fd7126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:29.956821 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a66860-7661-4f59-b3ef-93e7d4fd7126 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:29.976075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 739/1461] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:32.243085 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31deca0d-6982-4193-adfd-099c525a7683 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:32.243695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31deca0d-6982-4193-adfd-099c525a7683 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:32.243695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31deca0d-6982-4193-adfd-099c525a7683 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:32.245008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 723/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:32.251791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ba75-e661-418c-8958-fc3f2e1018cd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:32.252545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ba75-e661-418c-8958-fc3f2e1018cd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:32.252712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb4ba75-e661-418c-8958-fc3f2e1018cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:32.443323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5eb4ba75-e661-418c-8958-fc3f2e1018cd None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:32.444529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5eb4ba75-e661-418c-8958-fc3f2e1018cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:32.484371 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:32.547701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 740/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:32.834746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31deca0d-6982-4193-adfd-099c525a7683 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:32.836438 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-31deca0d-6982-4193-adfd-099c525a7683 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:32.949391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31deca0d-6982-4193-adfd-099c525a7683 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:32.993484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:33.042348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5ebf92-27be-4129-8a29-eecfc5694f1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:33.042961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ebf92-27be-4129-8a29-eecfc5694f1a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:33.043233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ebf92-27be-4129-8a29-eecfc5694f1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:33.043502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5ebf92-27be-4129-8a29-eecfc5694f1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:33.045229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4a5ebf92-27be-4129-8a29-eecfc5694f1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:33.175467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ebf92-27be-4129-8a29-eecfc5694f1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:33.180443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5ebf92-27be-4129-8a29-eecfc5694f1a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:33.201160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 724/1464] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:35.391530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91af62-7625-4a69-976a-46c41c6540e4 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:35.392460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91af62-7625-4a69-976a-46c41c6540e4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:35.392460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd91af62-7625-4a69-976a-46c41c6540e4 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:35.394684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 741/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:35.403839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a721a00-0f2a-48e0-bb2c-0c5e2b25294a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:35.404274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a721a00-0f2a-48e0-bb2c-0c5e2b25294a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:35.404380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a721a00-0f2a-48e0-bb2c-0c5e2b25294a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:35.487645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9a721a00-0f2a-48e0-bb2c-0c5e2b25294a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:35.488849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9a721a00-0f2a-48e0-bb2c-0c5e2b25294a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:35.507716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:35.591903 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 725/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:35.880484 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91af62-7625-4a69-976a-46c41c6540e4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:35.883063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fd91af62-7625-4a69-976a-46c41c6540e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:36.000902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd91af62-7625-4a69-976a-46c41c6540e4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:36.072906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:36.080278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0290cb-e6c4-40f4-9a50-a2d9eae5edc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:36.080891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0290cb-e6c4-40f4-9a50-a2d9eae5edc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:36.081174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0290cb-e6c4-40f4-9a50-a2d9eae5edc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:36.081436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0290cb-e6c4-40f4-9a50-a2d9eae5edc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:36.083131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fe0290cb-e6c4-40f4-9a50-a2d9eae5edc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:36.196133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0290cb-e6c4-40f4-9a50-a2d9eae5edc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:36.197503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0290cb-e6c4-40f4-9a50-a2d9eae5edc2 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:36.215635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 742/1467] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:38.380886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc956e9b-c64b-4097-8b85-a4a72b36ea61 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:38.381697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc956e9b-c64b-4097-8b85-a4a72b36ea61 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:38.381697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc956e9b-c64b-4097-8b85-a4a72b36ea61 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:38.383813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 726/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:55:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:38.389236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cd61f-65c2-4ba9-b0ae-37af6ecbf633 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:38.389705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cd61f-65c2-4ba9-b0ae-37af6ecbf633 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:38.389825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083cd61f-65c2-4ba9-b0ae-37af6ecbf633 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:38.456946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-083cd61f-65c2-4ba9-b0ae-37af6ecbf633 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:38.458177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-083cd61f-65c2-4ba9-b0ae-37af6ecbf633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:38.508225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:38.562995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 743/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:55:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:38.849870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc956e9b-c64b-4097-8b85-a4a72b36ea61 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:38.852373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fc956e9b-c64b-4097-8b85-a4a72b36ea61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:38.972238 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc956e9b-c64b-4097-8b85-a4a72b36ea61 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:55:39.024995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:39.062450 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c751c4-6a2c-4ef0-97de-62b2aa16aa60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:55:39.063109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c751c4-6a2c-4ef0-97de-62b2aa16aa60 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:39.063382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c751c4-6a2c-4ef0-97de-62b2aa16aa60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:39.063683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c751c4-6a2c-4ef0-97de-62b2aa16aa60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:39.065325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-83c751c4-6a2c-4ef0-97de-62b2aa16aa60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:39.185072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c751c4-6a2c-4ef0-97de-62b2aa16aa60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:55:39.186651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c751c4-6a2c-4ef0-97de-62b2aa16aa60 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:55:39.205660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 727/1470] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:55:48.561181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=22 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:49.203332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=66 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:59.181814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bba43-2ce7-4f6f-943f-5dc3b7d610a8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:59.182695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bba43-2ce7-4f6f-943f-5dc3b7d610a8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:59.182805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5bba43-2ce7-4f6f-943f-5dc3b7d610a8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:59.184175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 744/1471] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:59.190285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8166677-2129-47f6-98fc-915cc82b1964 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:59.190769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8166677-2129-47f6-98fc-915cc82b1964 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:59.190882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8166677-2129-47f6-98fc-915cc82b1964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:59.269418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e8166677-2129-47f6-98fc-915cc82b1964 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:59.270746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e8166677-2129-47f6-98fc-915cc82b1964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:59.380120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 728/1472] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:55:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:59.689855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd4e34-deaa-4c0c-b121-277ad8edcb8f None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:59.690134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd4e34-deaa-4c0c-b121-277ad8edcb8f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:59.690444 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bd4e34-deaa-4c0c-b121-277ad8edcb8f None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:59.691812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 745/1473] 10.222.0.25 () {54 vars in 966 bytes} [Tue Sep 13 06:55:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:55:59.698695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a88ff-92e0-4449-9032-e4af5aba55cb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:55:59.698981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a88ff-92e0-4449-9032-e4af5aba55cb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:55:59.699737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94a88ff-92e0-4449-9032-e4af5aba55cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:55:59.786887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a94a88ff-92e0-4449-9032-e4af5aba55cb None None] MFA Rules not processed for user `3505c8965c4a42e493a2f5916cab56dc`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:55:59.788106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a94a88ff-92e0-4449-9032-e4af5aba55cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:55:59.813762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:55:59.873260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 729/1474] 10.222.0.25 () {58 vars in 1063 bytes} [Tue Sep 13 06:55:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:55:59.881234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd4e34-deaa-4c0c-b121-277ad8edcb8f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:55:59.882776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-56bd4e34-deaa-4c0c-b121-277ad8edcb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:00.007809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bd4e34-deaa-4c0c-b121-277ad8edcb8f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:56:00.087290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2325f365-3e41-43b8-a858-721960088269 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3505c8965c4a42e493a2f5916cab56dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:56:00.088138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325f365-3e41-43b8-a858-721960088269 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:00.088424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325f365-3e41-43b8-a858-721960088269 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:00.088700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2325f365-3e41-43b8-a858-721960088269 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:00.091344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2325f365-3e41-43b8-a858-721960088269 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:00.149202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:00.205305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325f365-3e41-43b8-a858-721960088269 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:56:00.206797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2325f365-3e41-43b8-a858-721960088269 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:56:00.226656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 746/1475] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:55:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:56:02.457423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ddad4-b0aa-4a71-8b58-b6f5047b479e None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:02.458435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ddad4-b0aa-4a71-8b58-b6f5047b479e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:02.458435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ddad4-b0aa-4a71-8b58-b6f5047b479e None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:02.460563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 730/1476] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:56:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:56:02.468370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66af3-c92e-4fd0-8e50-9babbeeac2a0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:02.468831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66af3-c92e-4fd0-8e50-9babbeeac2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:02.468936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a66af3-c92e-4fd0-8e50-9babbeeac2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:02.549366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-76a66af3-c92e-4fd0-8e50-9babbeeac2a0 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:56:02.550634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-76a66af3-c92e-4fd0-8e50-9babbeeac2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:02.652865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 747/1477] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:56:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:56:02.926415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ddad4-b0aa-4a71-8b58-b6f5047b479e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:56:02.928017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-748ddad4-b0aa-4a71-8b58-b6f5047b479e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:03.066204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ddad4-b0aa-4a71-8b58-b6f5047b479e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:56:03.122330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:03.171635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03860346-5d3f-4511-8f4f-29ab3678fd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:56:03.172259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03860346-5d3f-4511-8f4f-29ab3678fd9b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:03.172542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03860346-5d3f-4511-8f4f-29ab3678fd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:03.172814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03860346-5d3f-4511-8f4f-29ab3678fd9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:03.173463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03860346-5d3f-4511-8f4f-29ab3678fd9b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:56:03.175251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-03860346-5d3f-4511-8f4f-29ab3678fd9b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 06:56:03.175251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 13 06:56:03.175251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: Sep 13 06:56:03.176404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03860346-5d3f-4511-8f4f-29ab3678fd9b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:56:03.196518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 731/1478] 10.222.0.25 () {56 vars in 1108 bytes} [Tue Sep 13 06:56:02 2022] GET /identity/v3/projects => generated 2553 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 06:56:05.375612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b1c52-711d-416d-8b11-692aaeefb3de None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:05.376485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b1c52-711d-416d-8b11-692aaeefb3de None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:05.376485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9b1c52-711d-416d-8b11-692aaeefb3de None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:05.378617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 748/1479] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:56:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:56:05.385152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbd3b8-f18d-49b7-ab0a-585ca881f5ac None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:05.385679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbd3b8-f18d-49b7-ab0a-585ca881f5ac None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:05.385679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afbd3b8-f18d-49b7-ab0a-585ca881f5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:05.462183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7afbd3b8-f18d-49b7-ab0a-585ca881f5ac None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:56:05.463343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7afbd3b8-f18d-49b7-ab0a-585ca881f5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:05.571011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 732/1480] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:56:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:56:05.865288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b1c52-711d-416d-8b11-692aaeefb3de None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:56:05.866929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7b9b1c52-711d-416d-8b11-692aaeefb3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:05.882466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:06.040332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9b1c52-711d-416d-8b11-692aaeefb3de None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:56:06.127976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:06.167574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cc8fb44-e309-476a-b23b-2bf89ba0bbcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:56:06.168157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8fb44-e309-476a-b23b-2bf89ba0bbcd service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:06.168430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8fb44-e309-476a-b23b-2bf89ba0bbcd service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:06.168701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cc8fb44-e309-476a-b23b-2bf89ba0bbcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:06.170373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9cc8fb44-e309-476a-b23b-2bf89ba0bbcd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:06.281240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8fb44-e309-476a-b23b-2bf89ba0bbcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:56:06.282881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cc8fb44-e309-476a-b23b-2bf89ba0bbcd service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:56:06.303077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 749/1481] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:56:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:56:08.631503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac01e0-0553-4c82-bb19-1e6e10893d95 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:08.632019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac01e0-0553-4c82-bb19-1e6e10893d95 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:08.632019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ac01e0-0553-4c82-bb19-1e6e10893d95 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:08.633399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 733/1482] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:56:08.639218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2bff4-d81c-458d-bca7-b48a519961b8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:08.639710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2bff4-d81c-458d-bca7-b48a519961b8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:08.639710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d2bff4-d81c-458d-bca7-b48a519961b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:08.706503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-50d2bff4-d81c-458d-bca7-b48a519961b8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:56:08.707705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-50d2bff4-d81c-458d-bca7-b48a519961b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:08.754813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:08.808019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 750/1483] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:56:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:56:09.093172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac01e0-0553-4c82-bb19-1e6e10893d95 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:56:09.094766 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e4ac01e0-0553-4c82-bb19-1e6e10893d95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:09.120637 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:09.206932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ac01e0-0553-4c82-bb19-1e6e10893d95 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:56:09.288105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fdf8cd-c67b-42a2-b9e4-70a93b14ff5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:56:09.288733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdf8cd-c67b-42a2-b9e4-70a93b14ff5c service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:09.289011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdf8cd-c67b-42a2-b9e4-70a93b14ff5c service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:09.289285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fdf8cd-c67b-42a2-b9e4-70a93b14ff5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:09.290995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-39fdf8cd-c67b-42a2-b9e4-70a93b14ff5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:09.371990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:09.409930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdf8cd-c67b-42a2-b9e4-70a93b14ff5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:56:09.411409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fdf8cd-c67b-42a2-b9e4-70a93b14ff5c service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:56:09.429832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 734/1484] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:56:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:56:11.743155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4920a-a7dc-4d5a-8145-646c5b7a7672 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:11.743805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4920a-a7dc-4d5a-8145-646c5b7a7672 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:11.743805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e4920a-a7dc-4d5a-8145-646c5b7a7672 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:11.745605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 751/1485] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:56:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:56:11.755812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548094fd-927c-48e1-a208-d899da1915e5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:11.756632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548094fd-927c-48e1-a208-d899da1915e5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:11.756632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548094fd-927c-48e1-a208-d899da1915e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:11.831565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-548094fd-927c-48e1-a208-d899da1915e5 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:56:11.832791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-548094fd-927c-48e1-a208-d899da1915e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:11.936883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 735/1486] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:56:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:56:12.231153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4920a-a7dc-4d5a-8145-646c5b7a7672 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:56:12.233876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-10e4920a-a7dc-4d5a-8145-646c5b7a7672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:12.342377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10e4920a-a7dc-4d5a-8145-646c5b7a7672 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:56:12.428077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:12.429608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f363246-eda8-40dd-b803-2b951eab862c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:56:12.430391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f363246-eda8-40dd-b803-2b951eab862c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:12.430890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f363246-eda8-40dd-b803-2b951eab862c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:12.431170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f363246-eda8-40dd-b803-2b951eab862c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:12.434014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0f363246-eda8-40dd-b803-2b951eab862c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:12.548792 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f363246-eda8-40dd-b803-2b951eab862c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:56:12.550173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f363246-eda8-40dd-b803-2b951eab862c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:56:12.569714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 752/1487] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:56:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:56:14.835532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b6e4-5ceb-4383-9742-07c777e4ef71 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:14.836513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b6e4-5ceb-4383-9742-07c777e4ef71 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:14.836513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826b6e4-5ceb-4383-9742-07c777e4ef71 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:14.838640 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 736/1488] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:56:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:56:14.844292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b3b94-3bcc-4725-85ce-e81b8979ff5a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:14.844766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b3b94-3bcc-4725-85ce-e81b8979ff5a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:14.844862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016b3b94-3bcc-4725-85ce-e81b8979ff5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:14.913331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-016b3b94-3bcc-4725-85ce-e81b8979ff5a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:56:14.914500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-016b3b94-3bcc-4725-85ce-e81b8979ff5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:14.969440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:15.014767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 753/1489] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:56:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:56:15.308905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b6e4-5ceb-4383-9742-07c777e4ef71 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:56:15.311131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d826b6e4-5ceb-4383-9742-07c777e4ef71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:15.348409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:15.434553 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826b6e4-5ceb-4383-9742-07c777e4ef71 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:56:15.515502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3513034-fbd9-4e41-82dc-84fb2d490d76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:56:15.516115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3513034-fbd9-4e41-82dc-84fb2d490d76 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:15.516413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3513034-fbd9-4e41-82dc-84fb2d490d76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:15.516676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3513034-fbd9-4e41-82dc-84fb2d490d76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:15.518357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e3513034-fbd9-4e41-82dc-84fb2d490d76 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:15.606887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:15.638588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3513034-fbd9-4e41-82dc-84fb2d490d76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:56:15.640051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3513034-fbd9-4e41-82dc-84fb2d490d76 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:56:15.658455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 737/1490] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:56:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:56:18.018158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eed94-59d0-4330-b6a1-d6080614df87 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:18.019031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eed94-59d0-4330-b6a1-d6080614df87 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:18.019031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775eed94-59d0-4330-b6a1-d6080614df87 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:18.021171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 754/1491] 10.222.0.25 () {54 vars in 927 bytes} [Tue Sep 13 06:56:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:56:18.027604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fc69d-436b-432d-93ac-b692087a5437 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:18.028108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fc69d-436b-432d-93ac-b692087a5437 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:18.028317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7fc69d-436b-432d-93ac-b692087a5437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:18.097632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5b7fc69d-436b-432d-93ac-b692087a5437 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:56:18.098875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5b7fc69d-436b-432d-93ac-b692087a5437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:18.216866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 738/1492] 10.222.0.25 () {58 vars in 1024 bytes} [Tue Sep 13 06:56:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:56:18.510114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eed94-59d0-4330-b6a1-d6080614df87 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:56:18.511666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-775eed94-59d0-4330-b6a1-d6080614df87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:18.622033 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775eed94-59d0-4330-b6a1-d6080614df87 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:56:18.703706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca11213-1bf7-46f6-b183-8d502a70a7ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:56:18.704563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca11213-1bf7-46f6-b183-8d502a70a7ba service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:56:18.704920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca11213-1bf7-46f6-b183-8d502a70a7ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:56:18.705273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca11213-1bf7-46f6-b183-8d502a70a7ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:56:18.707908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3ca11213-1bf7-46f6-b183-8d502a70a7ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:56:18.721909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:18.828767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca11213-1bf7-46f6-b183-8d502a70a7ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:56:18.830253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca11213-1bf7-46f6-b183-8d502a70a7ba service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:56:18.848520 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 755/1493] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:56:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:56:28.215099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=82 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:56:28.846116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=50 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:32.422892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed0073-34d3-487d-8dce-7190f65f6bd3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:32.423556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed0073-34d3-487d-8dce-7190f65f6bd3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:32.423556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ed0073-34d3-487d-8dce-7190f65f6bd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:32.512045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-80ed0073-34d3-487d-8dce-7190f65f6bd3 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:57:32.513265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80ed0073-34d3-487d-8dce-7190f65f6bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:32.624011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 739/1494] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 06:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:57:32.632241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354b0ad-7647-4c65-a834-42295b58fbe9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:32.632811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354b0ad-7647-4c65-a834-42295b58fbe9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:32.632811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8354b0ad-7647-4c65-a834-42295b58fbe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:32.716386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8354b0ad-7647-4c65-a834-42295b58fbe9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:57:32.717754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8354b0ad-7647-4c65-a834-42295b58fbe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:32.826898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 756/1495] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 06:57:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 06:57:32.832775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed0073-34d3-487d-8dce-7190f65f6bd3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:32.834646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-80ed0073-34d3-487d-8dce-7190f65f6bd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:32.853124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ed0073-34d3-487d-8dce-7190f65f6bd3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:32.921661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:32.958109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:32.958742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:32.959016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:32.959266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:32.960043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 06:57:32.960342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:32.961399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 06:57:32.961399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 13 06:57:32.961399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: Sep 13 06:57:32.962320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:32.974167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 740/1496] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 06:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:57:32.983332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354b0ad-7647-4c65-a834-42295b58fbe9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:32.997706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8354b0ad-7647-4c65-a834-42295b58fbe9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:33.067603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:33.103928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:33.104538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:33.104808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:33.105066 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:33.105688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:33.107695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 06:57:33.107695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 13 06:57:33.107695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: Sep 13 06:57:33.108878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:33.169548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 757/1497] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 06:57:32 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:57:33.173975 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:33.175521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:33.209590 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb68fcf-049f-4d0c-873a-e4bc7ea2b356 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:33.312105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:33.312711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:33.312990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:33.313259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:33.313870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:33.314997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 06:57:33.314997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 13 06:57:33.314997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: Sep 13 06:57:33.316230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:33.365810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 741/1498] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 06:57:33 2022] POST /identity/v3/users => generated 290 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 06:57:33.370045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:33.371703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:33.434850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db0f2e-3b39-4934-be18-d299fb518a8f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:33.518154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:33.529076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:33.529684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:33.529960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:33.530274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:33.530910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:33.531933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 06:57:33.531933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 13 06:57:33.531933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: Sep 13 06:57:33.532813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:33.545306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 758/1499] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 06:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 06:57:33.549529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:33.551119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:33.563570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00de58-c164-4456-9ce2-2505dd1301a8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:33.591137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:33.664078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea778ed2-8745-4a7b-a63f-15e188927221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:33.664747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea778ed2-8745-4a7b-a63f-15e188927221 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:33.665070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea778ed2-8745-4a7b-a63f-15e188927221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:33.665334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea778ed2-8745-4a7b-a63f-15e188927221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:33.666046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea778ed2-8745-4a7b-a63f-15e188927221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:33.667194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-ea778ed2-8745-4a7b-a63f-15e188927221 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 06:57:33.667194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 13 06:57:33.667194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: Sep 13 06:57:33.668129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea778ed2-8745-4a7b-a63f-15e188927221 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:33.681293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 742/1500] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 06:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 06:57:33.686166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:33.687735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:33.725159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cf5c52-dbd0-43c9-839d-6730a914547f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:33.832742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1270fd22-7414-440f-9f7c-59fd866d50aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:33.833574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270fd22-7414-440f-9f7c-59fd866d50aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:33.834006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270fd22-7414-440f-9f7c-59fd866d50aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:33.834475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1270fd22-7414-440f-9f7c-59fd866d50aa admin admin] PATH_INFO: `/v3/projects/d726a5601a474078a58e4842f6922eae/users/04c6131b6d864f53ab781b107c371bdb/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:33.878539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270fd22-7414-440f-9f7c-59fd866d50aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d726a5601a474078a58e4842f6922eae, user_id=04c6131b6d864f53ab781b107c371bdb, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:33.879611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-1270fd22-7414-440f-9f7c-59fd866d50aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 06:57:33.879611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 13 06:57:33.879611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: Sep 13 06:57:33.881556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1270fd22-7414-440f-9f7c-59fd866d50aa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:33.917246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 759/1501] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 06:57:33 2022] PUT /identity/v3/projects/d726a5601a474078a58e4842f6922eae/users/04c6131b6d864f53ab781b107c371bdb/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:57:33.930394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abf2de-77b6-4b39-885c-a7d2b779b4d4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:33.930685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abf2de-77b6-4b39-885c-a7d2b779b4d4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:33.930943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86abf2de-77b6-4b39-885c-a7d2b779b4d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:33.992535 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-86abf2de-77b6-4b39-885c-a7d2b779b4d4 None None] MFA Rules not processed for user `04c6131b6d864f53ab781b107c371bdb`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:57:33.993744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-86abf2de-77b6-4b39-885c-a7d2b779b4d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:34.078939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 743/1502] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 06:57:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:57:34.092764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3acb7-6a65-46c7-9cfc-bc78a0aa9096 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:34.093050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3acb7-6a65-46c7-9cfc-bc78a0aa9096 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:34.093323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd3acb7-6a65-46c7-9cfc-bc78a0aa9096 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:34.094803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 760/1503] 10.222.0.25 () {54 vars in 964 bytes} [Tue Sep 13 06:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:57:34.099740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b438d1a-0fa8-4f47-b823-c8b0c51fa1ef None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:34.100031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b438d1a-0fa8-4f47-b823-c8b0c51fa1ef None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:34.100307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b438d1a-0fa8-4f47-b823-c8b0c51fa1ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:34.133894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:34.183194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5b438d1a-0fa8-4f47-b823-c8b0c51fa1ef None None] MFA Rules not processed for user `88520ce6493a44f39b19c67592f03632`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:57:34.184427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5b438d1a-0fa8-4f47-b823-c8b0c51fa1ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:34.273035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 744/1504] 10.222.0.25 () {58 vars in 1061 bytes} [Tue Sep 13 06:57:34 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:57:34.281221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3acb7-6a65-46c7-9cfc-bc78a0aa9096 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:34.282789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9bd3acb7-6a65-46c7-9cfc-bc78a0aa9096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:34.336373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:34.393521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd3acb7-6a65-46c7-9cfc-bc78a0aa9096 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:34.471549 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:34.472153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:34.472433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:34.472703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:34.474340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:34.599247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:34.600730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:34.604241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:34.621686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 761/1505] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:57:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:57:34.686099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733ddb8-b734-44ff-b2e0-9357d397cb0a None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:34.686581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733ddb8-b734-44ff-b2e0-9357d397cb0a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:34.686685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b733ddb8-b734-44ff-b2e0-9357d397cb0a None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:34.688029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 745/1506] 10.222.0.25 () {52 vars in 830 bytes} [Tue Sep 13 06:57:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:57:34.691734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:34.692955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:34.813687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ff382f-0047-4053-a150-9b49ad98164e service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:34.915774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:34.916314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:34.916588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:34.916854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] PATH_INFO: `/v3/users/04c6131b6d864f53ab781b107c371bdb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:34.929945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] RBAC: Authorizing `identity:delete_user(user_id=04c6131b6d864f53ab781b107c371bdb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:34.930998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 06:57:34.930998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 13 06:57:34.930998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: Sep 13 06:57:34.932542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:34.942401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:35.002237 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04c6131b6d864f53ab781b107c371bdb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:35.002765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04c6131b6d864f53ab781b107c371bdb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:35.013874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04c6131b6d864f53ab781b107c371bdb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:35.045034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04c6131b6d864f53ab781b107c371bdb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:35.045740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04c6131b6d864f53ab781b107c371bdb'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:35.069976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 762/1507] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 06:57:34 2022] DELETE /identity/v3/users/04c6131b6d864f53ab781b107c371bdb => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:57:35.077270 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce066919-03e0-4bdd-9584-911bedd05227 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:35.077707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce066919-03e0-4bdd-9584-911bedd05227 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:35.078171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce066919-03e0-4bdd-9584-911bedd05227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:35.145778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ce066919-03e0-4bdd-9584-911bedd05227 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:57:35.147023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ce066919-03e0-4bdd-9584-911bedd05227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:35.245907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:35.260275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 746/1508] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 06:57:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:57:35.270694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:35.271917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:35.380569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777f354-25d5-4b76-b3b0-39117100002d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:35.441959 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:35.470133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091a85d-d326-45e0-9515-5e2a8e661313 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:35.470757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091a85d-d326-45e0-9515-5e2a8e661313 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:35.471070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091a85d-d326-45e0-9515-5e2a8e661313 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:35.471331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091a85d-d326-45e0-9515-5e2a8e661313 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:35.473023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d091a85d-d326-45e0-9515-5e2a8e661313 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:35.596747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091a85d-d326-45e0-9515-5e2a8e661313 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:35.598215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091a85d-d326-45e0-9515-5e2a8e661313 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:35.617628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 763/1509] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:57:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:57:36.037283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce066919-03e0-4bdd-9584-911bedd05227 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:57:36.038593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ce066919-03e0-4bdd-9584-911bedd05227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:57:36.049897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce066919-03e0-4bdd-9584-911bedd05227 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:57:36.159104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:57:36.159736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:57:36.160010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:57:36.160281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] PATH_INFO: `/v3/projects/d726a5601a474078a58e4842f6922eae` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:57:36.172455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d726a5601a474078a58e4842f6922eae)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:57:36.173491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 06:57:36.173491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 13 06:57:36.173491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: Sep 13 06:57:36.174743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:57:36.240743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd726a5601a474078a58e4842f6922eae', 'request_id': 'req-61067f35-e79b-483a-bc2f-d2d98dc23131'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:36.254826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd726a5601a474078a58e4842f6922eae', 'request_id': 'req-61067f35-e79b-483a-bc2f-d2d98dc23131'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:36.261709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd726a5601a474078a58e4842f6922eae', 'request_id': 'req-61067f35-e79b-483a-bc2f-d2d98dc23131'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:36.290259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] The token cache is being invalidate because project d726a5601a474078a58e4842f6922eae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 06:57:36.290480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:36.291004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61067f35-e79b-483a-bc2f-d2d98dc23131 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 06:57:36.292435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 747/1510] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 06:57:36 2022] DELETE /identity/v3/projects/d726a5601a474078a58e4842f6922eae => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 06:57:45.615072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=62 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:57:46.288171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:59:03.476531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedda84-75d0-40ac-ba5d-47f2f26f33cd None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:03.477440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedda84-75d0-40ac-ba5d-47f2f26f33cd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:03.477440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dedda84-75d0-40ac-ba5d-47f2f26f33cd None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:03.479559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 764/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:59:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:59:03.488143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379129f-cb41-4e6b-96d8-fad94c472481 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:03.488951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379129f-cb41-4e6b-96d8-fad94c472481 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:03.488951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379129f-cb41-4e6b-96d8-fad94c472481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:03.573784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0379129f-cb41-4e6b-96d8-fad94c472481 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:03.575018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0379129f-cb41-4e6b-96d8-fad94c472481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:03.688183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 748/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:59:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:03.843777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedda84-75d0-40ac-ba5d-47f2f26f33cd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:59:03.847460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0dedda84-75d0-40ac-ba5d-47f2f26f33cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:03.983020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dedda84-75d0-40ac-ba5d-47f2f26f33cd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:59:03.989304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629baeb1-c2e8-4d46-9aaf-024b9dd1ee8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:59:03.989916 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629baeb1-c2e8-4d46-9aaf-024b9dd1ee8d service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:03.990222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629baeb1-c2e8-4d46-9aaf-024b9dd1ee8d service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:03.990500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629baeb1-c2e8-4d46-9aaf-024b9dd1ee8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:03.999179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629baeb1-c2e8-4d46-9aaf-024b9dd1ee8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:59:04.000774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-629baeb1-c2e8-4d46-9aaf-024b9dd1ee8d service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:59:04.010093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 765/1513] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:59:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:59:05.091042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f80f7-949f-4f85-a65a-eb80fc6f7c64 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:05.091978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f80f7-949f-4f85-a65a-eb80fc6f7c64 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:05.091978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9f80f7-949f-4f85-a65a-eb80fc6f7c64 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:05.093920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 749/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:59:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:59:05.104208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e4617-e670-4abc-b6b0-cf0d7d5dac6c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:05.105013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e4617-e670-4abc-b6b0-cf0d7d5dac6c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:05.105192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e4617-e670-4abc-b6b0-cf0d7d5dac6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:05.131123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-807e4617-e670-4abc-b6b0-cf0d7d5dac6c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:05.132328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-807e4617-e670-4abc-b6b0-cf0d7d5dac6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:05.141855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 766/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:59:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:05.293948 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f80f7-949f-4f85-a65a-eb80fc6f7c64 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:59:05.312678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9f80f7-949f-4f85-a65a-eb80fc6f7c64 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:59:05.319416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c268f548-1be7-4a53-af8e-a500e23a5b0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:59:05.320370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268f548-1be7-4a53-af8e-a500e23a5b0b service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:05.320828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268f548-1be7-4a53-af8e-a500e23a5b0b service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:05.321382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c268f548-1be7-4a53-af8e-a500e23a5b0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:05.330060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268f548-1be7-4a53-af8e-a500e23a5b0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:59:05.331709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c268f548-1be7-4a53-af8e-a500e23a5b0b service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:59:05.341615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 750/1516] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:59:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:59:05.525607 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7441993-c034-420a-a521-6783b6cb3061 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:05.525868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7441993-c034-420a-a521-6783b6cb3061 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:05.526305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7441993-c034-420a-a521-6783b6cb3061 None None] PATH_INFO: `/v3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:05.527612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 767/1517] 10.222.0.23 () {54 vars in 925 bytes} [Tue Sep 13 06:59:05 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 06:59:05.536320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a9240-f3f1-4d09-a7e3-41e03461f38f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:05.536601 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a9240-f3f1-4d09-a7e3-41e03461f38f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:05.536871 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a9240-f3f1-4d09-a7e3-41e03461f38f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:05.606416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b13a9240-f3f1-4d09-a7e3-41e03461f38f None None] MFA Rules not processed for user `e8202ad6007246078c4d25365dc5a0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:05.607546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b13a9240-f3f1-4d09-a7e3-41e03461f38f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:05.624848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:59:05.681534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 751/1518] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Sep 13 06:59:05 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:14.392775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c30ee-1d56-41e7-96e8-a7a8df06fae5 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:14.393450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c30ee-1d56-41e7-96e8-a7a8df06fae5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:14.393450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889c30ee-1d56-41e7-96e8-a7a8df06fae5 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:14.394752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 768/1519] 10.222.0.25 () {54 vars in 961 bytes} [Tue Sep 13 06:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:59:14.399998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd351f-ddd1-4d1e-b31b-be414cbe17a0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:14.400441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd351f-ddd1-4d1e-b31b-be414cbe17a0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:14.400562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd351f-ddd1-4d1e-b31b-be414cbe17a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:14.421009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-adbd351f-ddd1-4d1e-b31b-be414cbe17a0 None None] MFA Rules not processed for user `e8202ad6007246078c4d25365dc5a0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:14.422271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-adbd351f-ddd1-4d1e-b31b-be414cbe17a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:14.432409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 752/1520] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Sep 13 06:59:14 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:14.440039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c30ee-1d56-41e7-96e8-a7a8df06fae5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:59:14.454121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889c30ee-1d56-41e7-96e8-a7a8df06fae5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:59:14.459820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2679150c-cdcd-4f47-9b15-3487cc71329c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:59:14.460423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679150c-cdcd-4f47-9b15-3487cc71329c service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:14.460704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679150c-cdcd-4f47-9b15-3487cc71329c service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:14.460975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2679150c-cdcd-4f47-9b15-3487cc71329c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:14.470005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679150c-cdcd-4f47-9b15-3487cc71329c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:59:14.471643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2679150c-cdcd-4f47-9b15-3487cc71329c service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:59:14.481974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 769/1521] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:59:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 06:59:16.851311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed55f7-8651-4b96-9fab-27bd4a1cc501 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:16.851968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed55f7-8651-4b96-9fab-27bd4a1cc501 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:16.851968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed55f7-8651-4b96-9fab-27bd4a1cc501 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:16.853213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 753/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:59:16.861001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f284c-962a-4932-8765-ce34c03fe3d6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:16.861813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f284c-962a-4932-8765-ce34c03fe3d6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:16.861813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570f284c-962a-4932-8765-ce34c03fe3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:16.886109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-570f284c-962a-4932-8765-ce34c03fe3d6 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:16.887336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-570f284c-962a-4932-8765-ce34c03fe3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:16.896820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 770/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:59:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:17.054953 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed55f7-8651-4b96-9fab-27bd4a1cc501 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:59:17.071352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed55f7-8651-4b96-9fab-27bd4a1cc501 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:59:17.077152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b984ffe-154a-48bd-8cc3-268a5dd9c4f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:59:17.077760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b984ffe-154a-48bd-8cc3-268a5dd9c4f8 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:17.078040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b984ffe-154a-48bd-8cc3-268a5dd9c4f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:17.078349 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b984ffe-154a-48bd-8cc3-268a5dd9c4f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:17.086989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b984ffe-154a-48bd-8cc3-268a5dd9c4f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:59:17.088658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b984ffe-154a-48bd-8cc3-268a5dd9c4f8 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:59:17.098365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 754/1524] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:59:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:59:18.169933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e477-b0bd-48d3-901a-89ff24262fce None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:18.170968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e477-b0bd-48d3-901a-89ff24262fce None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:18.170968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e477-b0bd-48d3-901a-89ff24262fce None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:18.173060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 771/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Sep 13 06:59:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:59:18.180383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ad18b-513f-4f2d-b8a0-0bb7246b8851 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:18.180866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ad18b-513f-4f2d-b8a0-0bb7246b8851 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:18.180981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28ad18b-513f-4f2d-b8a0-0bb7246b8851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:18.202780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b28ad18b-513f-4f2d-b8a0-0bb7246b8851 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:18.204006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b28ad18b-513f-4f2d-b8a0-0bb7246b8851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:18.213340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 755/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Sep 13 06:59:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:18.365110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e477-b0bd-48d3-901a-89ff24262fce None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:59:18.383267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e477-b0bd-48d3-901a-89ff24262fce None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:59:18.389635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfd3f00-1e40-43b6-8e1e-48163510e3a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:59:18.390266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd3f00-1e40-43b6-8e1e-48163510e3a8 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:18.390542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd3f00-1e40-43b6-8e1e-48163510e3a8 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:18.390813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfd3f00-1e40-43b6-8e1e-48163510e3a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:18.399990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd3f00-1e40-43b6-8e1e-48163510e3a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:59:18.401735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfd3f00-1e40-43b6-8e1e-48163510e3a8 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:59:18.411177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 772/1527] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:59:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:59:22.424197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173c914-f228-4b76-b9cc-6f6afd9c53e9 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:22.424860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173c914-f228-4b76-b9cc-6f6afd9c53e9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:22.424860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c173c914-f228-4b76-b9cc-6f6afd9c53e9 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:22.426118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 756/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:59:22.432815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6690c7-2bd6-412d-83d7-4801e6437a0a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:22.433301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6690c7-2bd6-412d-83d7-4801e6437a0a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:22.433419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6690c7-2bd6-412d-83d7-4801e6437a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:22.466435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1e6690c7-2bd6-412d-83d7-4801e6437a0a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:22.468193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1e6690c7-2bd6-412d-83d7-4801e6437a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:22.480391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 773/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:59:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:22.767891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173c914-f228-4b76-b9cc-6f6afd9c53e9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:59:22.771027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c173c914-f228-4b76-b9cc-6f6afd9c53e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:22.865387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c173c914-f228-4b76-b9cc-6f6afd9c53e9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:59:22.869408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c203ed32-6a29-4f1e-b54f-001055971a6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:59:22.870202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203ed32-6a29-4f1e-b54f-001055971a6e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:22.870681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203ed32-6a29-4f1e-b54f-001055971a6e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:22.870780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c203ed32-6a29-4f1e-b54f-001055971a6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:22.880904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203ed32-6a29-4f1e-b54f-001055971a6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:59:22.882420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c203ed32-6a29-4f1e-b54f-001055971a6e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:59:22.892059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 757/1530] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:59:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:59:25.018985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abbdeb-701f-41a2-adab-ac0ee6bf9c44 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:25.020032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abbdeb-701f-41a2-adab-ac0ee6bf9c44 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:25.020032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48abbdeb-701f-41a2-adab-ac0ee6bf9c44 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:25.022319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 774/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:59:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:59:25.027972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29784fe9-0bec-4e60-b388-2b40800c0590 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:25.028471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29784fe9-0bec-4e60-b388-2b40800c0590 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:25.028471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29784fe9-0bec-4e60-b388-2b40800c0590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:25.050234 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-29784fe9-0bec-4e60-b388-2b40800c0590 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:25.051453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-29784fe9-0bec-4e60-b388-2b40800c0590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:25.060975 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 758/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:59:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:25.343726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48abbdeb-701f-41a2-adab-ac0ee6bf9c44 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:59:25.359317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48abbdeb-701f-41a2-adab-ac0ee6bf9c44 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:59:25.363364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77d533f-6915-4769-b22b-255d29843faa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:59:25.364004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d533f-6915-4769-b22b-255d29843faa service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:25.364284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d533f-6915-4769-b22b-255d29843faa service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:25.364555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77d533f-6915-4769-b22b-255d29843faa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:25.375213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d533f-6915-4769-b22b-255d29843faa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:59:25.376702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77d533f-6915-4769-b22b-255d29843faa service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:59:25.387238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 775/1533] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:59:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:59:29.188507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b86da-1c9c-4b1c-8745-8dd5712e7eb3 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:29.189101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b86da-1c9c-4b1c-8745-8dd5712e7eb3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:29.189101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075b86da-1c9c-4b1c-8745-8dd5712e7eb3 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:29.190470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 759/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:59:29.196976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83d696-046b-421f-a0b7-92d0740d17ce None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:29.197708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83d696-046b-421f-a0b7-92d0740d17ce None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:29.197857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca83d696-046b-421f-a0b7-92d0740d17ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:29.206507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:59:29.219694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ca83d696-046b-421f-a0b7-92d0740d17ce None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:29.220898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ca83d696-046b-421f-a0b7-92d0740d17ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:29.229497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 776/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:59:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:29.532992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b86da-1c9c-4b1c-8745-8dd5712e7eb3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:59:29.535054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-075b86da-1c9c-4b1c-8745-8dd5712e7eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:29.604258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:59:29.619831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075b86da-1c9c-4b1c-8745-8dd5712e7eb3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:59:29.624334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa3439-148d-4ac9-88c7-bd4273977a76 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:59:29.624952 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa3439-148d-4ac9-88c7-bd4273977a76 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:29.625236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa3439-148d-4ac9-88c7-bd4273977a76 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:29.625481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa3439-148d-4ac9-88c7-bd4273977a76 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:29.635725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa3439-148d-4ac9-88c7-bd4273977a76 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:59:29.637295 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa3439-148d-4ac9-88c7-bd4273977a76 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:59:29.647088 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 760/1536] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:59:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:59:31.801764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5311e-db53-4234-8c86-0c8f0b0183c0 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:31.802715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5311e-db53-4234-8c86-0c8f0b0183c0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:31.802715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de5311e-db53-4234-8c86-0c8f0b0183c0 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:31.804731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 777/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Sep 13 06:59:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 06:59:31.810849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae09e01-2cbd-4033-ade9-258ff0aed18e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:31.811312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae09e01-2cbd-4033-ade9-258ff0aed18e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:31.811415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae09e01-2cbd-4033-ade9-258ff0aed18e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:31.832443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2ae09e01-2cbd-4033-ade9-258ff0aed18e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 06:59:31.833647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2ae09e01-2cbd-4033-ade9-258ff0aed18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:31.842116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 761/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Sep 13 06:59:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 06:59:32.148766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5311e-db53-4234-8c86-0c8f0b0183c0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 06:59:32.152650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5de5311e-db53-4234-8c86-0c8f0b0183c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 06:59:32.172950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de5311e-db53-4234-8c86-0c8f0b0183c0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 06:59:32.177235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a7de3-58d4-48a2-b671-26ffd400246d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 06:59:32.177841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a7de3-58d4-48a2-b671-26ffd400246d service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 06:59:32.178119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a7de3-58d4-48a2-b671-26ffd400246d service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 06:59:32.178424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a7de3-58d4-48a2-b671-26ffd400246d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 06:59:32.191266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a7de3-58d4-48a2-b671-26ffd400246d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 06:59:32.193585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208a7de3-58d4-48a2-b671-26ffd400246d service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 06:59:32.205177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 778/1539] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 06:59:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 06:59:41.827469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=11 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 06:59:42.200993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=8 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:54.117669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b8e1b-c1d3-4fa1-af21-a0bbf588060a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.118393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b8e1b-c1d3-4fa1-af21-a0bbf588060a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.118393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7b8e1b-c1d3-4fa1-af21-a0bbf588060a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.144587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4d7b8e1b-c1d3-4fa1-af21-a0bbf588060a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.145984 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4d7b8e1b-c1d3-4fa1-af21-a0bbf588060a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.159388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 762/1540] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:00:54.169328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1fee2-e894-4285-8790-0ffbdd11df4c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.169967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1fee2-e894-4285-8790-0ffbdd11df4c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.169967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e1fee2-e894-4285-8790-0ffbdd11df4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.195979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a5e1fee2-e894-4285-8790-0ffbdd11df4c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.197375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a5e1fee2-e894-4285-8790-0ffbdd11df4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.210387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 779/1541] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:54.215686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b8e1b-c1d3-4fa1-af21-a0bbf588060a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:54.226483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7b8e1b-c1d3-4fa1-af21-a0bbf588060a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:54.238346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905b5a-c5c2-4846-8bc0-dbe5c74fedc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:54.238997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905b5a-c5c2-4846-8bc0-dbe5c74fedc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.239277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905b5a-c5c2-4846-8bc0-dbe5c74fedc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.239550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90905b5a-c5c2-4846-8bc0-dbe5c74fedc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.240214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905b5a-c5c2-4846-8bc0-dbe5c74fedc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:00:54.240662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905b5a-c5c2-4846-8bc0-dbe5c74fedc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:54.242355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90905b5a-c5c2-4846-8bc0-dbe5c74fedc8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:54.259832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 763/1542] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:00:54.269030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1fee2-e894-4285-8790-0ffbdd11df4c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:54.279577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e1fee2-e894-4285-8790-0ffbdd11df4c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:54.289248 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec12f3-35e9-4777-91ab-6f7cf4ed0809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:54.289891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec12f3-35e9-4777-91ab-6f7cf4ed0809 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.290193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec12f3-35e9-4777-91ab-6f7cf4ed0809 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.290469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec12f3-35e9-4777-91ab-6f7cf4ed0809 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.291153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec12f3-35e9-4777-91ab-6f7cf4ed0809 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:54.293053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec12f3-35e9-4777-91ab-6f7cf4ed0809 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:54.353790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 780/1543] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/projects => generated 385 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:54.358714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905b5a-c5c2-4846-8bc0-dbe5c74fedc8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:54.369889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90905b5a-c5c2-4846-8bc0-dbe5c74fedc8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:54.378411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe50c2-d07d-45f0-b85d-f4ebc290e781 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.378881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe50c2-d07d-45f0-b85d-f4ebc290e781 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.379295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe50c2-d07d-45f0-b85d-f4ebc290e781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.421909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fafe50c2-d07d-45f0-b85d-f4ebc290e781 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.423343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fafe50c2-d07d-45f0-b85d-f4ebc290e781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.509339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b9401-ee4b-4ba2-b505-001720786770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:54.510305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b9401-ee4b-4ba2-b505-001720786770 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.510592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b9401-ee4b-4ba2-b505-001720786770 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.510891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b9401-ee4b-4ba2-b505-001720786770 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.511517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b9401-ee4b-4ba2-b505-001720786770 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:54.514581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b9401-ee4b-4ba2-b505-001720786770 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:54.531941 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 781/1544] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:00:54.548986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ac95-b39f-4099-8461-4b319e96d9a5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.549339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ac95-b39f-4099-8461-4b319e96d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.549545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e12ac95-b39f-4099-8461-4b319e96d9a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.572963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 764/1545] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/users => generated 293 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:54.585554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b9401-ee4b-4ba2-b505-001720786770 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:54.602179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9e12ac95-b39f-4099-8461-4b319e96d9a5 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.602179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9e12ac95-b39f-4099-8461-4b319e96d9a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.609173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b9401-ee4b-4ba2-b505-001720786770 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:54.622356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 782/1546] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:00:54.628101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7ff01-df10-43f9-a14a-6179fee7426c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:54.628748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7ff01-df10-43f9-a14a-6179fee7426c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.629072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7ff01-df10-43f9-a14a-6179fee7426c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.629294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a7ff01-df10-43f9-a14a-6179fee7426c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.629948 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7ff01-df10-43f9-a14a-6179fee7426c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:54.631600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a7ff01-df10-43f9-a14a-6179fee7426c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:54.634448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1b15-54e2-4c39-b9ab-135a8aa33f56 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.634728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1b15-54e2-4c39-b9ab-135a8aa33f56 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.635000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336b1b15-54e2-4c39-b9ab-135a8aa33f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.671341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 765/1547] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:54.672237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7ff01-df10-43f9-a14a-6179fee7426c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:54.684400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-336b1b15-54e2-4c39-b9ab-135a8aa33f56 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.685949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-336b1b15-54e2-4c39-b9ab-135a8aa33f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.692314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a7ff01-df10-43f9-a14a-6179fee7426c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:54.701485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 783/1548] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:54.702797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67856d7-4888-40bf-85c4-c8f7a17121e7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.703502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67856d7-4888-40bf-85c4-c8f7a17121e7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.703797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67856d7-4888-40bf-85c4-c8f7a17121e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.703927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025bf02-2566-4641-81b0-b6ff50a73da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:54.704674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025bf02-2566-4641-81b0-b6ff50a73da9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.704959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025bf02-2566-4641-81b0-b6ff50a73da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.705230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3025bf02-2566-4641-81b0-b6ff50a73da9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.705928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025bf02-2566-4641-81b0-b6ff50a73da9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:00:54.706268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025bf02-2566-4641-81b0-b6ff50a73da9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:54.708559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3025bf02-2566-4641-81b0-b6ff50a73da9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:54.727052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 766/1549] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:00:54.728095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025bf02-2566-4641-81b0-b6ff50a73da9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:54.728711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f67856d7-4888-40bf-85c4-c8f7a17121e7 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.730155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f67856d7-4888-40bf-85c4-c8f7a17121e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.739826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3025bf02-2566-4641-81b0-b6ff50a73da9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:54.742366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 784/1550] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:54.743835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635803ec-020e-4784-9e84-695ba8745411 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.744260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635803ec-020e-4784-9e84-695ba8745411 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.744687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635803ec-020e-4784-9e84-695ba8745411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.751581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1df0d80-b2e4-4685-b3bb-0f6be2bb12b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:54.752215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df0d80-b2e4-4685-b3bb-0f6be2bb12b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.752499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df0d80-b2e4-4685-b3bb-0f6be2bb12b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.753262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1df0d80-b2e4-4685-b3bb-0f6be2bb12b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.754112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df0d80-b2e4-4685-b3bb-0f6be2bb12b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:54.757244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1df0d80-b2e4-4685-b3bb-0f6be2bb12b5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:54.767975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-635803ec-020e-4784-9e84-695ba8745411 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.769292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-635803ec-020e-4784-9e84-695ba8745411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.771787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 767/1551] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:54 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:54.773248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972e643-8a38-4ec9-a0f9-5703d1907ae1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.773705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972e643-8a38-4ec9-a0f9-5703d1907ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.774274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5972e643-8a38-4ec9-a0f9-5703d1907ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.778110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 785/1552] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:54.779407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0129dd-2a6a-443a-b346-9407702a0307 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.779682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0129dd-2a6a-443a-b346-9407702a0307 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.779951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0129dd-2a6a-443a-b346-9407702a0307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.801886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5972e643-8a38-4ec9-a0f9-5703d1907ae1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.802575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ec0129dd-2a6a-443a-b346-9407702a0307 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.803635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5972e643-8a38-4ec9-a0f9-5703d1907ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.803782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ec0129dd-2a6a-443a-b346-9407702a0307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.814214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 786/1553] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:54.815088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0129dd-2a6a-443a-b346-9407702a0307 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:54.815560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 768/1554] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:54.816972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012e095-e2e7-4474-b779-b7d02d6ad325 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.817428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012e095-e2e7-4474-b779-b7d02d6ad325 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.817876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e012e095-e2e7-4474-b779-b7d02d6ad325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.829855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0129dd-2a6a-443a-b346-9407702a0307 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:54.834575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194f0e9b-3f32-48e6-aee2-84b91a14491b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:54.835485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f0e9b-3f32-48e6-aee2-84b91a14491b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.835628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f0e9b-3f32-48e6-aee2-84b91a14491b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.835831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194f0e9b-3f32-48e6-aee2-84b91a14491b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.836538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f0e9b-3f32-48e6-aee2-84b91a14491b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:54.838403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194f0e9b-3f32-48e6-aee2-84b91a14491b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:54.840868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e012e095-e2e7-4474-b779-b7d02d6ad325 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.842166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e012e095-e2e7-4474-b779-b7d02d6ad325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.850898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 769/1555] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:54.851948 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012e095-e2e7-4474-b779-b7d02d6ad325 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:54.861891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e012e095-e2e7-4474-b779-b7d02d6ad325 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:54.867310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a8a58-be44-41a6-97e3-b08d692bff45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:54.867972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a8a58-be44-41a6-97e3-b08d692bff45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.868250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a8a58-be44-41a6-97e3-b08d692bff45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.868460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 787/1556] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/projects => generated 391 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:54.868570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a8a58-be44-41a6-97e3-b08d692bff45 admin admin] PATH_INFO: `/v3/projects/8c7dc05c1c9d40b2bb0defc9dd157d6e/users/5a0fd7347cad41bc80f427f872cf78fa/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.869808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14184-91b1-4bd2-bd6f-4ae7814d6586 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.870112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14184-91b1-4bd2-bd6f-4ae7814d6586 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.870410 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e14184-91b1-4bd2-bd6f-4ae7814d6586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.884377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a8a58-be44-41a6-97e3-b08d692bff45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c7dc05c1c9d40b2bb0defc9dd157d6e, user_id=5a0fd7347cad41bc80f427f872cf78fa, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:54.885912 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-ad6a8a58-be44-41a6-97e3-b08d692bff45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:00:54.885912 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 13 07:00:54.885912 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: Sep 13 07:00:54.887974 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a8a58-be44-41a6-97e3-b08d692bff45 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:54.890886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e3e14184-91b1-4bd2-bd6f-4ae7814d6586 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.892119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e3e14184-91b1-4bd2-bd6f-4ae7814d6586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.905660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 770/1557] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:54 2022] PUT /identity/v3/projects/8c7dc05c1c9d40b2bb0defc9dd157d6e/users/5a0fd7347cad41bc80f427f872cf78fa/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:54.905660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a8a58-be44-41a6-97e3-b08d692bff45 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:54.956096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 788/1558] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:54.957392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad029a6-7008-4cc1-b124-e4b65a1f3970 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.957680 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad029a6-7008-4cc1-b124-e4b65a1f3970 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.957953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad029a6-7008-4cc1-b124-e4b65a1f3970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.964680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:54.968750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a8a58-be44-41a6-97e3-b08d692bff45 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:54.975777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833a8910-6b40-4840-9877-fdd9ce8512f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:54.976401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a8910-6b40-4840-9877-fdd9ce8512f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.976687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a8910-6b40-4840-9877-fdd9ce8512f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.976966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833a8910-6b40-4840-9877-fdd9ce8512f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.977609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a8910-6b40-4840-9877-fdd9ce8512f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:00:54.977718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7ad029a6-7008-4cc1-b124-e4b65a1f3970 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:54.977916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a8910-6b40-4840-9877-fdd9ce8512f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:54.978913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7ad029a6-7008-4cc1-b124-e4b65a1f3970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:54.979588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833a8910-6b40-4840-9877-fdd9ce8512f0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:54.987081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 789/1559] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:54.988427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161f11-319e-434e-877b-30a7849dbcfe None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.988714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161f11-319e-434e-877b-30a7849dbcfe None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.988986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70161f11-319e-434e-877b-30a7849dbcfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:54.991967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 771/1560] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:00:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:00:54.993435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299274e-9b4a-4d2a-9cb6-f535f64b7890 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:54.993909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299274e-9b4a-4d2a-9cb6-f535f64b7890 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:54.994398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0299274e-9b4a-4d2a-9cb6-f535f64b7890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.002962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:55.014605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0299274e-9b4a-4d2a-9cb6-f535f64b7890 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:55.015875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0299274e-9b4a-4d2a-9cb6-f535f64b7890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:55.017188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-70161f11-319e-434e-877b-30a7849dbcfe None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:55.018617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-70161f11-319e-434e-877b-30a7849dbcfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:55.024386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 772/1561] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:55.031114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 790/1562] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:55.031114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161f11-319e-434e-877b-30a7849dbcfe None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.031994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299274e-9b4a-4d2a-9cb6-f535f64b7890 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.038026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70161f11-319e-434e-877b-30a7849dbcfe None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.041899 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0299274e-9b4a-4d2a-9cb6-f535f64b7890 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.042636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba7ec4-cb80-4402-9292-396d05b25a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.043238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba7ec4-cb80-4402-9292-396d05b25a6d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.043508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba7ec4-cb80-4402-9292-396d05b25a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.043791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba7ec4-cb80-4402-9292-396d05b25a6d admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.044426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba7ec4-cb80-4402-9292-396d05b25a6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.045454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-deba7ec4-cb80-4402-9292-396d05b25a6d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:00:55.045454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 13 07:00:55.045454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: Sep 13 07:00:55.046695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba7ec4-cb80-4402-9292-396d05b25a6d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.046818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fab3ddc-6d80-4cb5-af04-16d709113571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.047400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3ddc-6d80-4cb5-af04-16d709113571 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.047678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3ddc-6d80-4cb5-af04-16d709113571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.047946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fab3ddc-6d80-4cb5-af04-16d709113571 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.048584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab3ddc-6d80-4cb5-af04-16d709113571 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:00:55.048885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab3ddc-6d80-4cb5-af04-16d709113571 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.051223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fab3ddc-6d80-4cb5-af04-16d709113571 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.064224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 773/1563] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:00:55.065491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb331a66-37a6-4970-9131-9ea1dd07ca82 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.066406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb331a66-37a6-4970-9131-9ea1dd07ca82 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.066689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb331a66-37a6-4970-9131-9ea1dd07ca82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.074508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 791/1564] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/users => generated 296 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.074508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba7ec4-cb80-4402-9292-396d05b25a6d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.084569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba7ec4-cb80-4402-9292-396d05b25a6d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.088128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eb331a66-37a6-4970-9131-9ea1dd07ca82 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:55.089387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eb331a66-37a6-4970-9131-9ea1dd07ca82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:55.089636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.090256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.090523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.090795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.091439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:00:55.091743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.092790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:00:55.092790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 13 07:00:55.092790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: Sep 13 07:00:55.093681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.097460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 774/1565] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:55.100828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196cc9e-af68-4eed-a338-cf7b22acb32f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.100828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196cc9e-af68-4eed-a338-cf7b22acb32f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.100828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196cc9e-af68-4eed-a338-cf7b22acb32f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.107886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 792/1566] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:00:55.109245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.119518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81055e5-b487-4f1c-bb9d-4b4bd7d09170 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.121254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2196cc9e-af68-4eed-a338-cf7b22acb32f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:55.122501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2196cc9e-af68-4eed-a338-cf7b22acb32f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:55.124331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0b73e-687a-44c9-9afb-9943cd66bb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.124959 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0b73e-687a-44c9-9afb-9943cd66bb7f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.125241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0b73e-687a-44c9-9afb-9943cd66bb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.125511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0b73e-687a-44c9-9afb-9943cd66bb7f admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.126184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0b73e-687a-44c9-9afb-9943cd66bb7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.128204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0b73e-687a-44c9-9afb-9943cd66bb7f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.131169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 775/1567] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:55.132312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2285420-2af3-4683-868e-f2f3cf214ef4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.132601 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2285420-2af3-4683-868e-f2f3cf214ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.132873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2285420-2af3-4683-868e-f2f3cf214ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.153157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f2285420-2af3-4683-868e-f2f3cf214ef4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:55.154418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f2285420-2af3-4683-868e-f2f3cf214ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:55.162436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 793/1568] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/projects => generated 389 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.162436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0b73e-687a-44c9-9afb-9943cd66bb7f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.162699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 776/1569] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:55.163343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2285420-2af3-4683-868e-f2f3cf214ef4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.172584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2285420-2af3-4683-868e-f2f3cf214ef4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.225357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0b73e-687a-44c9-9afb-9943cd66bb7f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.230342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba527b-0891-4631-ab8d-a1f8a3d43e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.231004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba527b-0891-4631-ab8d-a1f8a3d43e35 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.231236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c54415-528b-4eb0-b0d1-41e03504a1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.231380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba527b-0891-4631-ab8d-a1f8a3d43e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.231561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cba527b-0891-4631-ab8d-a1f8a3d43e35 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.231955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c54415-528b-4eb0-b0d1-41e03504a1e1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.232177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba527b-0891-4631-ab8d-a1f8a3d43e35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:00:55.232280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c54415-528b-4eb0-b0d1-41e03504a1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.232503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c54415-528b-4eb0-b0d1-41e03504a1e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.232613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba527b-0891-4631-ab8d-a1f8a3d43e35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.233194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c54415-528b-4eb0-b0d1-41e03504a1e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.234157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cba527b-0891-4631-ab8d-a1f8a3d43e35 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.234281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-40c54415-528b-4eb0-b0d1-41e03504a1e1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:00:55.234281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 13 07:00:55.234281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: Sep 13 07:00:55.235438 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c54415-528b-4eb0-b0d1-41e03504a1e1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.246114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 794/1570] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:00:55.247003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba527b-0891-4631-ab8d-a1f8a3d43e35 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.256971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cba527b-0891-4631-ab8d-a1f8a3d43e35 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.261836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c36a3-c5c0-41fa-bac9-f51cf87a34da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.262467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c36a3-c5c0-41fa-bac9-f51cf87a34da admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.262746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c36a3-c5c0-41fa-bac9-f51cf87a34da admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.263018 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3c36a3-c5c0-41fa-bac9-f51cf87a34da admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.263657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c36a3-c5c0-41fa-bac9-f51cf87a34da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.265255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3c36a3-c5c0-41fa-bac9-f51cf87a34da admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.267633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 777/1571] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.268530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c54415-528b-4eb0-b0d1-41e03504a1e1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.279410 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 795/1572] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:55.279410 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c36a3-c5c0-41fa-bac9-f51cf87a34da admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.287433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3c36a3-c5c0-41fa-bac9-f51cf87a34da admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.331693 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c54415-528b-4eb0-b0d1-41e03504a1e1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.335409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e2141-5251-4a76-9a72-f047b2ff895d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.336036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e2141-5251-4a76-9a72-f047b2ff895d service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.336317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e2141-5251-4a76-9a72-f047b2ff895d service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.336595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545e2141-5251-4a76-9a72-f047b2ff895d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.344851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eb3b2-4ea3-443b-9e1f-71133560d9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.345469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eb3b2-4ea3-443b-9e1f-71133560d9c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.345756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eb3b2-4ea3-443b-9e1f-71133560d9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.346028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34eb3b2-4ea3-443b-9e1f-71133560d9c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.346791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34eb3b2-4ea3-443b-9e1f-71133560d9c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.348705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34eb3b2-4ea3-443b-9e1f-71133560d9c3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.376242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 796/1573] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.377079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eb3b2-4ea3-443b-9e1f-71133560d9c3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.389193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:55.389774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:55.405157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e2141-5251-4a76-9a72-f047b2ff895d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.406593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545e2141-5251-4a76-9a72-f047b2ff895d service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.416475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 778/1574] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:00:55.417261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e2141-5251-4a76-9a72-f047b2ff895d service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.427609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545e2141-5251-4a76-9a72-f047b2ff895d service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.444323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34eb3b2-4ea3-443b-9e1f-71133560d9c3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.450953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739f161-fec3-4042-a29e-4fb9fb14f0a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.451645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739f161-fec3-4042-a29e-4fb9fb14f0a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.451930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739f161-fec3-4042-a29e-4fb9fb14f0a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.452204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739f161-fec3-4042-a29e-4fb9fb14f0a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.452878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6739f161-fec3-4042-a29e-4fb9fb14f0a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:00:55.453179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6739f161-fec3-4042-a29e-4fb9fb14f0a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.454964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6739f161-fec3-4042-a29e-4fb9fb14f0a0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.468271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 797/1575] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:00:55.468271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739f161-fec3-4042-a29e-4fb9fb14f0a0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.479465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739f161-fec3-4042-a29e-4fb9fb14f0a0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.484798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569241a-ed85-414c-8d6a-3bb9c672586f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.485408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569241a-ed85-414c-8d6a-3bb9c672586f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.485789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569241a-ed85-414c-8d6a-3bb9c672586f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.485979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7569241a-ed85-414c-8d6a-3bb9c672586f admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.487077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569241a-ed85-414c-8d6a-3bb9c672586f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:00:55.487077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569241a-ed85-414c-8d6a-3bb9c672586f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.488880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7569241a-ed85-414c-8d6a-3bb9c672586f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.502326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068617-6e75-45cc-bae5-c76dc6091881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.503771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068617-6e75-45cc-bae5-c76dc6091881 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.504122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068617-6e75-45cc-bae5-c76dc6091881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.504474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59068617-6e75-45cc-bae5-c76dc6091881 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.505333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59068617-6e75-45cc-bae5-c76dc6091881 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.506247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 798/1576] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:00:55.507398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569241a-ed85-414c-8d6a-3bb9c672586f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.507694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59068617-6e75-45cc-bae5-c76dc6091881 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.519539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7569241a-ed85-414c-8d6a-3bb9c672586f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.527545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f9ad5-d445-4508-81f5-ecf43bc87ddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.528344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f9ad5-d445-4508-81f5-ecf43bc87ddd admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.528685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f9ad5-d445-4508-81f5-ecf43bc87ddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.529056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66f9ad5-d445-4508-81f5-ecf43bc87ddd admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.529852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f9ad5-d445-4508-81f5-ecf43bc87ddd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.531871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66f9ad5-d445-4508-81f5-ecf43bc87ddd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.536016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 779/1577] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/users => generated 295 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.536016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068617-6e75-45cc-bae5-c76dc6091881 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.550849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59068617-6e75-45cc-bae5-c76dc6091881 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.556356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b288ae-0765-4d9d-b9b0-8730cc0ed1b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.557014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b288ae-0765-4d9d-b9b0-8730cc0ed1b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.557274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b288ae-0765-4d9d-b9b0-8730cc0ed1b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.557534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b288ae-0765-4d9d-b9b0-8730cc0ed1b7 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.558358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b288ae-0765-4d9d-b9b0-8730cc0ed1b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.560436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b288ae-0765-4d9d-b9b0-8730cc0ed1b7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.562113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 799/1578] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.566274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f9ad5-d445-4508-81f5-ecf43bc87ddd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.591887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 780/1579] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/users => generated 289 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.595104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b288ae-0765-4d9d-b9b0-8730cc0ed1b7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.614785 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b288ae-0765-4d9d-b9b0-8730cc0ed1b7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.656646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66f9ad5-d445-4508-81f5-ecf43bc87ddd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.661572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a4fd8-8cc5-4006-91d2-b6bc33acc599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.662254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a4fd8-8cc5-4006-91d2-b6bc33acc599 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.662522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a4fd8-8cc5-4006-91d2-b6bc33acc599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.662797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6a4fd8-8cc5-4006-91d2-b6bc33acc599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.663450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a4fd8-8cc5-4006-91d2-b6bc33acc599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.665085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6a4fd8-8cc5-4006-91d2-b6bc33acc599 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.676520 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 800/1580] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:55.677719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a4fd8-8cc5-4006-91d2-b6bc33acc599 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.688692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6a4fd8-8cc5-4006-91d2-b6bc33acc599 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.693488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fb225-6269-4a77-9b4a-533a4e8b7acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.694285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fb225-6269-4a77-9b4a-533a4e8b7acf admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.694540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b261c-f423-475a-851e-899cfe0fd8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.694674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fb225-6269-4a77-9b4a-533a4e8b7acf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.694864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2fb225-6269-4a77-9b4a-533a4e8b7acf admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.695517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b261c-f423-475a-851e-899cfe0fd8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.695648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fb225-6269-4a77-9b4a-533a4e8b7acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.695731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b261c-f423-475a-851e-899cfe0fd8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.695731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663b261c-f423-475a-851e-899cfe0fd8b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.696591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b261c-f423-475a-851e-899cfe0fd8b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.697946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2fb225-6269-4a77-9b4a-533a4e8b7acf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.698757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663b261c-f423-475a-851e-899cfe0fd8b7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.722722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 781/1581] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/users => generated 283 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.722722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fb225-6269-4a77-9b4a-533a4e8b7acf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.735761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2fb225-6269-4a77-9b4a-533a4e8b7acf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.737423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 801/1582] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/projects => generated 375 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.739026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b261c-f423-475a-851e-899cfe0fd8b7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.788705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:55.790129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:55.807072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be002c-4821-466d-afd8-c2e62cbe3915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.808050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be002c-4821-466d-afd8-c2e62cbe3915 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.808563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be002c-4821-466d-afd8-c2e62cbe3915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.808820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663b261c-f423-475a-851e-899cfe0fd8b7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.809376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55be002c-4821-466d-afd8-c2e62cbe3915 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.810384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be002c-4821-466d-afd8-c2e62cbe3915 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.813417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55be002c-4821-466d-afd8-c2e62cbe3915 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.814242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee21144-bda1-4c0a-bc97-9ccce91d7a1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.815810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee21144-bda1-4c0a-bc97-9ccce91d7a1d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.816298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee21144-bda1-4c0a-bc97-9ccce91d7a1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.817364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee21144-bda1-4c0a-bc97-9ccce91d7a1d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.822570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee21144-bda1-4c0a-bc97-9ccce91d7a1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.824846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee21144-bda1-4c0a-bc97-9ccce91d7a1d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.840214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 802/1583] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:55.841093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee21144-bda1-4c0a-bc97-9ccce91d7a1d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.852345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 782/1584] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/projects => generated 363 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.854738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee21144-bda1-4c0a-bc97-9ccce91d7a1d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.854857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be002c-4821-466d-afd8-c2e62cbe3915 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.919334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83bd45-b3a3-4a61-acc9-c197e74a1aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.920385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83bd45-b3a3-4a61-acc9-c197e74a1aa2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.920385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83bd45-b3a3-4a61-acc9-c197e74a1aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.920572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83bd45-b3a3-4a61-acc9-c197e74a1aa2 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.921222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83bd45-b3a3-4a61-acc9-c197e74a1aa2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.923391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83bd45-b3a3-4a61-acc9-c197e74a1aa2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.928600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55be002c-4821-466d-afd8-c2e62cbe3915 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.933934 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed111a61-49cf-4acf-b59a-cc22b7f7be4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.934787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed111a61-49cf-4acf-b59a-cc22b7f7be4c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.935052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed111a61-49cf-4acf-b59a-cc22b7f7be4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.935345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed111a61-49cf-4acf-b59a-cc22b7f7be4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.936308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed111a61-49cf-4acf-b59a-cc22b7f7be4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.938891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed111a61-49cf-4acf-b59a-cc22b7f7be4c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.949673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 803/1585] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/users => generated 287 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:55.950757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83bd45-b3a3-4a61-acc9-c197e74a1aa2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.955945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 783/1586] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:55.955945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed111a61-49cf-4acf-b59a-cc22b7f7be4c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.961629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83bd45-b3a3-4a61-acc9-c197e74a1aa2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.966307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed111a61-49cf-4acf-b59a-cc22b7f7be4c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:55.966478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c365ff-1c2c-4517-b4dd-2cd642eac263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.967129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c365ff-1c2c-4517-b4dd-2cd642eac263 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.967421 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c365ff-1c2c-4517-b4dd-2cd642eac263 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.967678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c365ff-1c2c-4517-b4dd-2cd642eac263 admin admin] PATH_INFO: `/v3/projects/38b6d7b146a34896bb8309e2c51348f8/users/0d8d0ff010c04d65b36ebe65c1cbbbfd/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.972071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bd6cd-478f-435e-bb55-d068c91f9bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:55.972789 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bd6cd-478f-435e-bb55-d068c91f9bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:55.973174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bd6cd-478f-435e-bb55-d068c91f9bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:55.973521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4bd6cd-478f-435e-bb55-d068c91f9bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:55.974314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bd6cd-478f-435e-bb55-d068c91f9bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.976006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4bd6cd-478f-435e-bb55-d068c91f9bf6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:55.985126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c365ff-1c2c-4517-b4dd-2cd642eac263 admin admin] RBAC: Authorizing `identity:create_grant(project_id=38b6d7b146a34896bb8309e2c51348f8, user_id=0d8d0ff010c04d65b36ebe65c1cbbbfd, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:55.989819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 784/1587] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:55.989819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bd6cd-478f-435e-bb55-d068c91f9bf6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:55.990055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c365ff-1c2c-4517-b4dd-2cd642eac263 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.004373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4bd6cd-478f-435e-bb55-d068c91f9bf6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.004727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 804/1588] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:55 2022] PUT /identity/v3/projects/38b6d7b146a34896bb8309e2c51348f8/users/0d8d0ff010c04d65b36ebe65c1cbbbfd/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:56.005712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c365ff-1c2c-4517-b4dd-2cd642eac263 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.072039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657537c1-7307-4575-b308-2d3abd31d7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.072802 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657537c1-7307-4575-b308-2d3abd31d7a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.073096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657537c1-7307-4575-b308-2d3abd31d7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.073380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657537c1-7307-4575-b308-2d3abd31d7a8 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.074055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657537c1-7307-4575-b308-2d3abd31d7a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.076552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657537c1-7307-4575-b308-2d3abd31d7a8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.091778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c365ff-1c2c-4517-b4dd-2cd642eac263 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.097120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49b2a4-9f1f-4f4b-aac5-4a5c5e362893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.097816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49b2a4-9f1f-4f4b-aac5-4a5c5e362893 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.098123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49b2a4-9f1f-4f4b-aac5-4a5c5e362893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.098417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc49b2a4-9f1f-4f4b-aac5-4a5c5e362893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.099124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49b2a4-9f1f-4f4b-aac5-4a5c5e362893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.102037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 785/1589] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:55 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:56.102284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc49b2a4-9f1f-4f4b-aac5-4a5c5e362893 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.106861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657537c1-7307-4575-b308-2d3abd31d7a8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.115600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 805/1590] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:56.116642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49b2a4-9f1f-4f4b-aac5-4a5c5e362893 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.118960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657537c1-7307-4575-b308-2d3abd31d7a8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.125669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675b7af-0f92-4fad-b12e-96a8d6416c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.127187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675b7af-0f92-4fad-b12e-96a8d6416c01 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.128293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675b7af-0f92-4fad-b12e-96a8d6416c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.128683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc49b2a4-9f1f-4f4b-aac5-4a5c5e362893 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.129043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9675b7af-0f92-4fad-b12e-96a8d6416c01 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.129951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675b7af-0f92-4fad-b12e-96a8d6416c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.132993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9675b7af-0f92-4fad-b12e-96a8d6416c01 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.133372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7746a4-7110-4ff5-b410-938ce3e308e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.133985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7746a4-7110-4ff5-b410-938ce3e308e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.134296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7746a4-7110-4ff5-b410-938ce3e308e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.134571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b7746a4-7110-4ff5-b410-938ce3e308e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.135275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7746a4-7110-4ff5-b410-938ce3e308e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.136914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b7746a4-7110-4ff5-b410-938ce3e308e9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.151739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 806/1591] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:56.151739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7746a4-7110-4ff5-b410-938ce3e308e9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.169022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 786/1592] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:56 2022] POST /identity/v3/users => generated 282 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:56.169022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675b7af-0f92-4fad-b12e-96a8d6416c01 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.169311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b7746a4-7110-4ff5-b410-938ce3e308e9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.175029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f285696-b270-404d-abee-b2bb6c724f24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.175912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f285696-b270-404d-abee-b2bb6c724f24 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.176141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f285696-b270-404d-abee-b2bb6c724f24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.176416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f285696-b270-404d-abee-b2bb6c724f24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.177632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f285696-b270-404d-abee-b2bb6c724f24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.179687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f285696-b270-404d-abee-b2bb6c724f24 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.182176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9675b7af-0f92-4fad-b12e-96a8d6416c01 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.187722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b137c-3817-4c0b-8d55-5e9190eaacd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.188426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b137c-3817-4c0b-8d55-5e9190eaacd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.188790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b137c-3817-4c0b-8d55-5e9190eaacd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.189763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b137c-3817-4c0b-8d55-5e9190eaacd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.189763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b137c-3817-4c0b-8d55-5e9190eaacd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.192325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661b137c-3817-4c0b-8d55-5e9190eaacd4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.197623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 807/1593] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:56.199001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a6cc1-9113-4bb8-9006-2bf8d4fd360f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.199287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a6cc1-9113-4bb8-9006-2bf8d4fd360f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.199568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a6cc1-9113-4bb8-9006-2bf8d4fd360f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.205299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 787/1594] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:56.206565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b137c-3817-4c0b-8d55-5e9190eaacd4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.226851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b137c-3817-4c0b-8d55-5e9190eaacd4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.233037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-605a6cc1-9113-4bb8-9006-2bf8d4fd360f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:56.233717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47da3-6120-4cab-b02d-abfd9a4aab29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.234314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-605a6cc1-9113-4bb8-9006-2bf8d4fd360f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:56.234533 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47da3-6120-4cab-b02d-abfd9a4aab29 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.234862 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47da3-6120-4cab-b02d-abfd9a4aab29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.235144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea47da3-6120-4cab-b02d-abfd9a4aab29 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.236021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47da3-6120-4cab-b02d-abfd9a4aab29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.239909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea47da3-6120-4cab-b02d-abfd9a4aab29 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.246478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 808/1595] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:56.246478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a6cc1-9113-4bb8-9006-2bf8d4fd360f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.255460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 788/1596] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:56.259297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47da3-6120-4cab-b02d-abfd9a4aab29 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.263239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a6cc1-9113-4bb8-9006-2bf8d4fd360f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.270146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836193c-66d4-4b83-a924-40bd07253e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.271455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836193c-66d4-4b83-a924-40bd07253e56 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.271967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836193c-66d4-4b83-a924-40bd07253e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.272444 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836193c-66d4-4b83-a924-40bd07253e56 admin admin] PATH_INFO: `/v3/projects/5d5ac5d4506943c48f4f7640d97f63c5/users/ecf2dfd9bbe54b3085e9638caf8ad044/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.273329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea47da3-6120-4cab-b02d-abfd9a4aab29 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.280298 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bf18d-80fa-4691-9dbd-688e4fc48aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.281121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bf18d-80fa-4691-9dbd-688e4fc48aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.281533 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bf18d-80fa-4691-9dbd-688e4fc48aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.281820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207bf18d-80fa-4691-9dbd-688e4fc48aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.283040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bf18d-80fa-4691-9dbd-688e4fc48aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.285499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207bf18d-80fa-4691-9dbd-688e4fc48aaf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.288343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:56.289431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:56.294296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836193c-66d4-4b83-a924-40bd07253e56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d5ac5d4506943c48f4f7640d97f63c5, user_id=ecf2dfd9bbe54b3085e9638caf8ad044, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.297902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c836193c-66d4-4b83-a924-40bd07253e56 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.302382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 789/1597] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:56.307845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bf18d-80fa-4691-9dbd-688e4fc48aaf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.322150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207bf18d-80fa-4691-9dbd-688e4fc48aaf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.325946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 809/1598] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:56 2022] PUT /identity/v3/projects/5d5ac5d4506943c48f4f7640d97f63c5/users/ecf2dfd9bbe54b3085e9638caf8ad044/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:56.325946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836193c-66d4-4b83-a924-40bd07253e56 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.398603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836193c-66d4-4b83-a924-40bd07253e56 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.405434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ab1d3-971e-4e16-8e79-ed35318e32bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.406134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab1d3-971e-4e16-8e79-ed35318e32bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.406428 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab1d3-971e-4e16-8e79-ed35318e32bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.406704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9ab1d3-971e-4e16-8e79-ed35318e32bc admin admin] PATH_INFO: `/v3/projects/5d415383e1c1471aae5712893d48c2b6/users/23088396d0ca47c5929690820a3f7e29/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.413144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b10211-f37f-49b6-b892-58e326d7d813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.414652 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b10211-f37f-49b6-b892-58e326d7d813 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.415013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b10211-f37f-49b6-b892-58e326d7d813 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.415352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b10211-f37f-49b6-b892-58e326d7d813 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.416069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b10211-f37f-49b6-b892-58e326d7d813 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.420082 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b10211-f37f-49b6-b892-58e326d7d813 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.426504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ab1d3-971e-4e16-8e79-ed35318e32bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d415383e1c1471aae5712893d48c2b6, user_id=23088396d0ca47c5929690820a3f7e29, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.429297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9ab1d3-971e-4e16-8e79-ed35318e32bc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.437069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 790/1599] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:56.437069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b10211-f37f-49b6-b892-58e326d7d813 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.447172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b10211-f37f-49b6-b892-58e326d7d813 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.456113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 810/1600] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:56 2022] PUT /identity/v3/projects/5d415383e1c1471aae5712893d48c2b6/users/23088396d0ca47c5929690820a3f7e29/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:56.456113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ab1d3-971e-4e16-8e79-ed35318e32bc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.456113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cf9ab1d3-971e-4e16-8e79-ed35318e32bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:56.528876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c38f-f5f2-490e-b1fe-d961947dfdb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.529397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9ab1d3-971e-4e16-8e79-ed35318e32bc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.530468 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c38f-f5f2-490e-b1fe-d961947dfdb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.530981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c38f-f5f2-490e-b1fe-d961947dfdb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.531487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc15c38f-f5f2-490e-b1fe-d961947dfdb7 admin admin] PATH_INFO: `/v3/projects/35fb3e3ada014f3aa7868201d2ce98d9/users/639d3e75244543f6b5f69cdddc7c4a2e/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.533311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc5a4-3fcb-4ba2-be58-27aac8a2d559 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.533929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc5a4-3fcb-4ba2-be58-27aac8a2d559 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.534502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc5a4-3fcb-4ba2-be58-27aac8a2d559 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.534817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0cc5a4-3fcb-4ba2-be58-27aac8a2d559 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.546622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc5a4-3fcb-4ba2-be58-27aac8a2d559 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.548199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0cc5a4-3fcb-4ba2-be58-27aac8a2d559 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.551293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c38f-f5f2-490e-b1fe-d961947dfdb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=35fb3e3ada014f3aa7868201d2ce98d9, user_id=639d3e75244543f6b5f69cdddc7c4a2e, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.555542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc15c38f-f5f2-490e-b1fe-d961947dfdb7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.563334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 811/1601] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:00:56.574236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc5a4-3fcb-4ba2-be58-27aac8a2d559 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.581187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 791/1602] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:56 2022] PUT /identity/v3/projects/35fb3e3ada014f3aa7868201d2ce98d9/users/639d3e75244543f6b5f69cdddc7c4a2e/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:56.582819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c38f-f5f2-490e-b1fe-d961947dfdb7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.599464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0cc5a4-3fcb-4ba2-be58-27aac8a2d559 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.732375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9387cee6-dd5a-4aa6-be45-30645be822ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.733685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387cee6-dd5a-4aa6-be45-30645be822ba admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.733685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387cee6-dd5a-4aa6-be45-30645be822ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.734374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9387cee6-dd5a-4aa6-be45-30645be822ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.735298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387cee6-dd5a-4aa6-be45-30645be822ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.736220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc15c38f-f5f2-490e-b1fe-d961947dfdb7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.736696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9387cee6-dd5a-4aa6-be45-30645be822ba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.755964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 812/1603] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:56.756160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f4760b-5c98-49ed-be5b-7e4300693429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.756281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601c6df-566f-4099-b823-9dba432c6552 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.756281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601c6df-566f-4099-b823-9dba432c6552 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.756459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4760b-5c98-49ed-be5b-7e4300693429 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.756544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5601c6df-566f-4099-b823-9dba432c6552 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.756631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4760b-5c98-49ed-be5b-7e4300693429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.756631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f4760b-5c98-49ed-be5b-7e4300693429 admin admin] PATH_INFO: `/v3/projects/b5ea0c5e37d54c28b948355d9c4be3f2/users/84f133a848ed4f64a2c1bd0fc53c0424/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.781361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4760b-5c98-49ed-be5b-7e4300693429 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5ea0c5e37d54c28b948355d9c4be3f2, user_id=84f133a848ed4f64a2c1bd0fc53c0424, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.784366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2f4760b-5c98-49ed-be5b-7e4300693429 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.787170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5601c6df-566f-4099-b823-9dba432c6552 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:56.791924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5601c6df-566f-4099-b823-9dba432c6552 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:56.804605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 813/1604] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:56.805565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601c6df-566f-4099-b823-9dba432c6552 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:56.815735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:56.821135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:56.822264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5601c6df-566f-4099-b823-9dba432c6552 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:56.829182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 792/1605] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:56 2022] PUT /identity/v3/projects/b5ea0c5e37d54c28b948355d9c4be3f2/users/84f133a848ed4f64a2c1bd0fc53c0424/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:56.831604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4dd8c-4ef4-4785-a64c-b8b0b06466e4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.831604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4dd8c-4ef4-4785-a64c-b8b0b06466e4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.831604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ef4dd8c-4ef4-4785-a64c-b8b0b06466e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.860995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7ef4dd8c-4ef4-4785-a64c-b8b0b06466e4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:56.862515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7ef4dd8c-4ef4-4785-a64c-b8b0b06466e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:56.937798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 793/1606] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:56.940444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11065c3c-0d2e-4798-baf7-df8b0b413785 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.940444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11065c3c-0d2e-4798-baf7-df8b0b413785 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.940444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11065c3c-0d2e-4798-baf7-df8b0b413785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.975040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f947f72-37c7-4f36-a876-44705b6218d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:56.977126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f947f72-37c7-4f36-a876-44705b6218d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:56.978017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f947f72-37c7-4f36-a876-44705b6218d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:56.978325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f947f72-37c7-4f36-a876-44705b6218d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:56.979036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f947f72-37c7-4f36-a876-44705b6218d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:56.980741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-11065c3c-0d2e-4798-baf7-df8b0b413785 None None] MFA Rules not processed for user `639d3e75244543f6b5f69cdddc7c4a2e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:56.983159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f947f72-37c7-4f36-a876-44705b6218d0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:56.983300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-11065c3c-0d2e-4798-baf7-df8b0b413785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:57.009788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 814/1607] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:56 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:57.010715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f947f72-37c7-4f36-a876-44705b6218d0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:57.031497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f947f72-37c7-4f36-a876-44705b6218d0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:57.050911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfc5ba-6cc0-43ba-a1c3-51136ca9dc45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:57.051615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfc5ba-6cc0-43ba-a1c3-51136ca9dc45 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.051890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfc5ba-6cc0-43ba-a1c3-51136ca9dc45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.052170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacfc5ba-6cc0-43ba-a1c3-51136ca9dc45 admin admin] PATH_INFO: `/v3/projects/250617554d634a3a95733753fc56512c/users/565e45e3594244a1a1233455fa913f70/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.078256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 794/1608] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:57.080039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac6a89-bf45-4c55-8b12-14ffa1d5d5d9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.080332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac6a89-bf45-4c55-8b12-14ffa1d5d5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.081087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac6a89-bf45-4c55-8b12-14ffa1d5d5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.097134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfc5ba-6cc0-43ba-a1c3-51136ca9dc45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=250617554d634a3a95733753fc56512c, user_id=565e45e3594244a1a1233455fa913f70, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:57.097134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacfc5ba-6cc0-43ba-a1c3-51136ca9dc45 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:57.133197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 815/1609] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:57 2022] PUT /identity/v3/projects/250617554d634a3a95733753fc56512c/users/565e45e3594244a1a1233455fa913f70/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:57.134755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfc5ba-6cc0-43ba-a1c3-51136ca9dc45 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:57.152754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3dac6a89-bf45-4c55-8b12-14ffa1d5d5d9 None None] MFA Rules not processed for user `84f133a848ed4f64a2c1bd0fc53c0424`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:57.154188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3dac6a89-bf45-4c55-8b12-14ffa1d5d5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:57.256476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacfc5ba-6cc0-43ba-a1c3-51136ca9dc45 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:57.263759 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc236cf-5f7d-405e-b1ad-2553c6617620 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:57.264725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc236cf-5f7d-405e-b1ad-2553c6617620 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.265072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc236cf-5f7d-405e-b1ad-2553c6617620 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.265509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc236cf-5f7d-405e-b1ad-2553c6617620 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.284459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 795/1610] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:57.285355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac6a89-bf45-4c55-8b12-14ffa1d5d5d9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:57.301188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac6a89-bf45-4c55-8b12-14ffa1d5d5d9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:57.305807 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da7587-83f9-4854-be9f-3b62b62fc6d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:57.307625 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da7587-83f9-4854-be9f-3b62b62fc6d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.308047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da7587-83f9-4854-be9f-3b62b62fc6d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.308328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55da7587-83f9-4854-be9f-3b62b62fc6d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.339266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:57.469874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc236cf-5f7d-405e-b1ad-2553c6617620 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:57.474782 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da7587-83f9-4854-be9f-3b62b62fc6d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:57.475156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc236cf-5f7d-405e-b1ad-2553c6617620 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:57.475592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55da7587-83f9-4854-be9f-3b62b62fc6d2 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:57.475592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 796/1611] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:00:57.475592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da7587-83f9-4854-be9f-3b62b62fc6d2 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:57.477658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 816/1612] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:00:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:00:57.479038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc236cf-5f7d-405e-b1ad-2553c6617620 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:57.480269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55da7587-83f9-4854-be9f-3b62b62fc6d2 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:57.492540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfef0f5-9b9a-49d3-86c8-b148a74f2ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:57.493433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfef0f5-9b9a-49d3-86c8-b148a74f2ade admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.493796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfef0f5-9b9a-49d3-86c8-b148a74f2ade admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.494169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfef0f5-9b9a-49d3-86c8-b148a74f2ade admin admin] PATH_INFO: `/v3/projects/fea098e26a964973a3f9db0e35ad3e24/users/3e9749655c0b48cb9841f7f1d4abd05e/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.551259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:57.555456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfef0f5-9b9a-49d3-86c8-b148a74f2ade admin admin] RBAC: Authorizing `identity:create_grant(project_id=fea098e26a964973a3f9db0e35ad3e24, user_id=3e9749655c0b48cb9841f7f1d4abd05e, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:57.558826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfef0f5-9b9a-49d3-86c8-b148a74f2ade admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:57.584439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 797/1613] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:57 2022] PUT /identity/v3/projects/fea098e26a964973a3f9db0e35ad3e24/users/3e9749655c0b48cb9841f7f1d4abd05e/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:57.585893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8c611-4eff-44ce-99a4-90949eb89dd2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.586268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8c611-4eff-44ce-99a4-90949eb89dd2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.586654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b8c611-4eff-44ce-99a4-90949eb89dd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.606603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc236cf-5f7d-405e-b1ad-2553c6617620 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:57.660236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-68b8c611-4eff-44ce-99a4-90949eb89dd2 None None] MFA Rules not processed for user `565e45e3594244a1a1233455fa913f70`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:57.661446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-68b8c611-4eff-44ce-99a4-90949eb89dd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:57.751259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f9bb5-5292-4efe-9d64-74a3af930ce5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:57.752544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f9bb5-5292-4efe-9d64-74a3af930ce5 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.756526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f9bb5-5292-4efe-9d64-74a3af930ce5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.756526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911f9bb5-5292-4efe-9d64-74a3af930ce5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.837570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 798/1614] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:57 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:57.841123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f9bb5-5292-4efe-9d64-74a3af930ce5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:57.842230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e948efa-8c1d-45a2-a577-632c26d94aa1 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.842528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e948efa-8c1d-45a2-a577-632c26d94aa1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.842844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e948efa-8c1d-45a2-a577-632c26d94aa1 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.844153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911f9bb5-5292-4efe-9d64-74a3af930ce5 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:57.848252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 799/1615] 10.222.0.25 () {54 vars in 962 bytes} [Tue Sep 13 07:00:57 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:00:57.849597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cd49e-048c-42b9-a16d-60e81babb6b3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.849892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cd49e-048c-42b9-a16d-60e81babb6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.855614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3cd49e-048c-42b9-a16d-60e81babb6b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.858320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 817/1616] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:00:57 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:00:57.859425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f9bb5-5292-4efe-9d64-74a3af930ce5 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:57.932586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3a3cd49e-048c-42b9-a16d-60e81babb6b3 None None] MFA Rules not processed for user `3e9749655c0b48cb9841f7f1d4abd05e`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:57.936626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3a3cd49e-048c-42b9-a16d-60e81babb6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:57.960132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911f9bb5-5292-4efe-9d64-74a3af930ce5 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:57.965186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72678f4e-55c2-425c-afae-b57d3d87b483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:57.966353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678f4e-55c2-425c-afae-b57d3d87b483 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:57.966886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678f4e-55c2-425c-afae-b57d3d87b483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:57.967257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72678f4e-55c2-425c-afae-b57d3d87b483 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:57.968027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72678f4e-55c2-425c-afae-b57d3d87b483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:57.970348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72678f4e-55c2-425c-afae-b57d3d87b483 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:58.014777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 818/1617] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:57 2022] POST /identity/v3/projects => generated 375 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:58.016320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349cd08-86a9-4bc9-a178-f812b487edb8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:58.016867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349cd08-86a9-4bc9-a178-f812b487edb8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:58.017798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2349cd08-86a9-4bc9-a178-f812b487edb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:58.090845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 800/1618] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:57 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:58.091874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cd49e-048c-42b9-a16d-60e81babb6b3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:58.110681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3cd49e-048c-42b9-a16d-60e81babb6b3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:58.128006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:58.134452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2349cd08-86a9-4bc9-a178-f812b487edb8 None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:58.135830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2349cd08-86a9-4bc9-a178-f812b487edb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:58.145866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:58.268492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2aaf3d-59a9-4fbc-929a-7bbf54d7aafa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:58.269443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2aaf3d-59a9-4fbc-929a-7bbf54d7aafa admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:58.269729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2aaf3d-59a9-4fbc-929a-7bbf54d7aafa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:58.270005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2aaf3d-59a9-4fbc-929a-7bbf54d7aafa admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:58.274119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2aaf3d-59a9-4fbc-929a-7bbf54d7aafa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:58.278858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2aaf3d-59a9-4fbc-929a-7bbf54d7aafa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:58.314542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 819/1619] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Sep 13 07:00:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 299 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:58.315700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349cd08-86a9-4bc9-a178-f812b487edb8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:58.333633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 801/1620] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:58 2022] POST /identity/v3/users => generated 294 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:58.333817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2349cd08-86a9-4bc9-a178-f812b487edb8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:58.345403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431aba2-7551-4590-9637-c327cf300e92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:58.346217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2aaf3d-59a9-4fbc-929a-7bbf54d7aafa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:58.374388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2aaf3d-59a9-4fbc-929a-7bbf54d7aafa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:58.374901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431aba2-7551-4590-9637-c327cf300e92 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:58.376337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431aba2-7551-4590-9637-c327cf300e92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:58.378219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6431aba2-7551-4590-9637-c327cf300e92 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:58.380987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431aba2-7551-4590-9637-c327cf300e92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:58.385239 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ab03f-cde0-4dbb-abbf-798a490bbd0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:58.386507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ab03f-cde0-4dbb-abbf-798a490bbd0c service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:58.387408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6431aba2-7551-4590-9637-c327cf300e92 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:58.387924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ab03f-cde0-4dbb-abbf-798a490bbd0c service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:58.388209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572ab03f-cde0-4dbb-abbf-798a490bbd0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:58.490585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 820/1621] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:00:58 2022] POST /identity/v3/projects => generated 365 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:58.493529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431aba2-7551-4590-9637-c327cf300e92 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:58.592176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ab03f-cde0-4dbb-abbf-798a490bbd0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:58.594858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572ab03f-cde0-4dbb-abbf-798a490bbd0c service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:58.627107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 802/1622] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:00:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:00:58.629195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ab03f-cde0-4dbb-abbf-798a490bbd0c service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:58.658207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572ab03f-cde0-4dbb-abbf-798a490bbd0c service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:58.685831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe386c-ede1-4098-92d4-1daf6a14b848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:58.687126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe386c-ede1-4098-92d4-1daf6a14b848 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:58.687622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe386c-ede1-4098-92d4-1daf6a14b848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:58.688696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe386c-ede1-4098-92d4-1daf6a14b848 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:58.691879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe386c-ede1-4098-92d4-1daf6a14b848 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:58.693436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6431aba2-7551-4590-9637-c327cf300e92 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:58.695802 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe386c-ede1-4098-92d4-1daf6a14b848 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:58.717738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38e46f-c559-483f-8ca6-44c2c4d80d50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:58.720488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38e46f-c559-483f-8ca6-44c2c4d80d50 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:58.720806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38e46f-c559-483f-8ca6-44c2c4d80d50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:58.721204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38e46f-c559-483f-8ca6-44c2c4d80d50 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:58.723518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38e46f-c559-483f-8ca6-44c2c4d80d50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:58.728748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38e46f-c559-483f-8ca6-44c2c4d80d50 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:58.757319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 821/1623] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:58.763124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38e46f-c559-483f-8ca6-44c2c4d80d50 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:58.763558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 803/1624] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:00:58 2022] POST /identity/v3/users => generated 289 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:00:58.771230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe386c-ede1-4098-92d4-1daf6a14b848 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:58.782264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38e46f-c559-483f-8ca6-44c2c4d80d50 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:58.800078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f21ac-317a-4e99-a34a-6094cb9062b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:58.800824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe386c-ede1-4098-92d4-1daf6a14b848 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:58.801015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f21ac-317a-4e99-a34a-6094cb9062b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:58.801129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f21ac-317a-4e99-a34a-6094cb9062b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:58.801391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3f21ac-317a-4e99-a34a-6094cb9062b2 admin admin] PATH_INFO: `/v3/projects/a64d84dec39f4257835f17fbbb2bd534/users/9716b1b6d02848d2ae6d530058216786/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:58.813737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766cb4-bcc0-43e7-84de-858672da2664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:58.814510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766cb4-bcc0-43e7-84de-858672da2664 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:58.815450 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766cb4-bcc0-43e7-84de-858672da2664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:58.818377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1766cb4-bcc0-43e7-84de-858672da2664 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:58.819559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1766cb4-bcc0-43e7-84de-858672da2664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:58.827249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1766cb4-bcc0-43e7-84de-858672da2664 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:58.847694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f21ac-317a-4e99-a34a-6094cb9062b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a64d84dec39f4257835f17fbbb2bd534, user_id=9716b1b6d02848d2ae6d530058216786, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:58.852573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3f21ac-317a-4e99-a34a-6094cb9062b2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:58.874764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 804/1625] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:58.878772 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766cb4-bcc0-43e7-84de-858672da2664 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:58.885554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 822/1626] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:58 2022] PUT /identity/v3/projects/a64d84dec39f4257835f17fbbb2bd534/users/9716b1b6d02848d2ae6d530058216786/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:58.889186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f21ac-317a-4e99-a34a-6094cb9062b2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:58.908229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1766cb4-bcc0-43e7-84de-858672da2664 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:59.014356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:59.022452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db052b3-6d0c-4616-b4ee-44ace38d97fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:59.022452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db052b3-6d0c-4616-b4ee-44ace38d97fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:59.022452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db052b3-6d0c-4616-b4ee-44ace38d97fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:59.022452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db052b3-6d0c-4616-b4ee-44ace38d97fe admin admin] PATH_INFO: `/v3/projects/474b37a67c7746eaad6c7e01d1474a2c/users/2ab9f38bc5ad464484b195bc4f20dece/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:59.031416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3f21ac-317a-4e99-a34a-6094cb9062b2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:59.049145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a5e33-954e-4970-ac88-94dec43f897e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:59.049825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a5e33-954e-4970-ac88-94dec43f897e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:59.050120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a5e33-954e-4970-ac88-94dec43f897e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:59.050401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a5e33-954e-4970-ac88-94dec43f897e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:59.051054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a5e33-954e-4970-ac88-94dec43f897e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:59.052876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a5e33-954e-4970-ac88-94dec43f897e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:59.067579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:59.074567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db052b3-6d0c-4616-b4ee-44ace38d97fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=474b37a67c7746eaad6c7e01d1474a2c, user_id=2ab9f38bc5ad464484b195bc4f20dece, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:59.074758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 823/1627] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:58 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:00:59.076594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a5e33-954e-4970-ac88-94dec43f897e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:59.085595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db052b3-6d0c-4616-b4ee-44ace38d97fe admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:59.104399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a5e33-954e-4970-ac88-94dec43f897e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:59.115647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e2ed9c1-2a7a-4b52-aa84-2958f6d00bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:59.116345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ed9c1-2a7a-4b52-aa84-2958f6d00bdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:59.116624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ed9c1-2a7a-4b52-aa84-2958f6d00bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:59.116901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e2ed9c1-2a7a-4b52-aa84-2958f6d00bdc admin admin] PATH_INFO: `/v3/projects/a64d84dec39f4257835f17fbbb2bd534/users/9716b1b6d02848d2ae6d530058216786/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:59.120135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 805/1628] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:58 2022] PUT /identity/v3/projects/474b37a67c7746eaad6c7e01d1474a2c/users/2ab9f38bc5ad464484b195bc4f20dece/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:59.121983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db052b3-6d0c-4616-b4ee-44ace38d97fe admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:59.123365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ed9c1-2a7a-4b52-aa84-2958f6d00bdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a64d84dec39f4257835f17fbbb2bd534, user_id=9716b1b6d02848d2ae6d530058216786, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:59.132314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e2ed9c1-2a7a-4b52-aa84-2958f6d00bdc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:59.166439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 824/1629] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:59 2022] PUT /identity/v3/projects/a64d84dec39f4257835f17fbbb2bd534/users/9716b1b6d02848d2ae6d530058216786/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:00:59.183813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70ed4c-5a8f-4bd3-a7a6-df3f1c8da89f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:59.184141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70ed4c-5a8f-4bd3-a7a6-df3f1c8da89f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:59.184429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d70ed4c-5a8f-4bd3-a7a6-df3f1c8da89f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:59.244431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5d70ed4c-5a8f-4bd3-a7a6-df3f1c8da89f None None] MFA Rules not processed for user `9716b1b6d02848d2ae6d530058216786`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:59.244431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5d70ed4c-5a8f-4bd3-a7a6-df3f1c8da89f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:59.258560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db052b3-6d0c-4616-b4ee-44ace38d97fe admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:59.409885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870b71-9285-4622-b170-5fdda4509371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:59.415128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870b71-9285-4622-b170-5fdda4509371 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:59.415547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870b71-9285-4622-b170-5fdda4509371 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:59.415925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d870b71-9285-4622-b170-5fdda4509371 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:59.416809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d870b71-9285-4622-b170-5fdda4509371 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:59.420826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d870b71-9285-4622-b170-5fdda4509371 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:59.444576 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 825/1630] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:59 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:00:59.457410 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70ed4c-5a8f-4bd3-a7a6-df3f1c8da89f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:59.477051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 806/1631] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:00:59 2022] GET /identity/v3/roles => generated 1420 bytes in 356 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:00:59.480994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870b71-9285-4622-b170-5fdda4509371 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:00:59.508488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d870b71-9285-4622-b170-5fdda4509371 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:59.532639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb7f537-ad37-4c2d-abde-0455ef3d6a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:59.534768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7f537-ad37-4c2d-abde-0455ef3d6a92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:59.535111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7f537-ad37-4c2d-abde-0455ef3d6a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:59.537036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb7f537-ad37-4c2d-abde-0455ef3d6a92 admin admin] PATH_INFO: `/v3/projects/474b37a67c7746eaad6c7e01d1474a2c/users/2ab9f38bc5ad464484b195bc4f20dece/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:59.544309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7f537-ad37-4c2d-abde-0455ef3d6a92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=474b37a67c7746eaad6c7e01d1474a2c, user_id=2ab9f38bc5ad464484b195bc4f20dece, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:59.563610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb7f537-ad37-4c2d-abde-0455ef3d6a92 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:59.607640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d70ed4c-5a8f-4bd3-a7a6-df3f1c8da89f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:00:59.640792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 807/1632] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:00:59 2022] PUT /identity/v3/projects/474b37a67c7746eaad6c7e01d1474a2c/users/2ab9f38bc5ad464484b195bc4f20dece/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:00:59.652726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf5c0c-8605-450a-a359-ef79f0c47280 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:59.653022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf5c0c-8605-450a-a359-ef79f0c47280 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:59.653302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bf5c0c-8605-450a-a359-ef79f0c47280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:59.725740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a7022-88fa-473b-8348-d0bd6df1bd01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:00:59.727724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a7022-88fa-473b-8348-d0bd6df1bd01 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:00:59.729393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a7022-88fa-473b-8348-d0bd6df1bd01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:00:59.729915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2a7022-88fa-473b-8348-d0bd6df1bd01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:00:59.782496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-25bf5c0c-8605-450a-a359-ef79f0c47280 None None] MFA Rules not processed for user `2ab9f38bc5ad464484b195bc4f20dece`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:00:59.783835 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-25bf5c0c-8605-450a-a359-ef79f0c47280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:00:59.820630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:00:59.881799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a7022-88fa-473b-8348-d0bd6df1bd01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:00:59.883496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2a7022-88fa-473b-8348-d0bd6df1bd01 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:00:59.906182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 826/1633] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:00:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:00.021014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 808/1634] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:00:59 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 370 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:00.035112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a7022-88fa-473b-8348-d0bd6df1bd01 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:00.053057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2a7022-88fa-473b-8348-d0bd6df1bd01 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:00.067060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f3bffe-709c-4974-bf16-3444dd70e667 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:00.067880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3bffe-709c-4974-bf16-3444dd70e667 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.068306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3bffe-709c-4974-bf16-3444dd70e667 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.068742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f3bffe-709c-4974-bf16-3444dd70e667 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.113542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3bffe-709c-4974-bf16-3444dd70e667 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:00.130661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f3bffe-709c-4974-bf16-3444dd70e667 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:00.144740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530896f-5e6e-460f-bb71-e1933df347d8 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.145007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530896f-5e6e-460f-bb71-e1933df347d8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.145289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530896f-5e6e-460f-bb71-e1933df347d8 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.146967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 809/1635] 10.222.0.25 () {54 vars in 962 bytes} [Tue Sep 13 07:01:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:01:00.157781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adb2c0-94d3-4cee-92b5-8194d35a4c8f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.158068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adb2c0-94d3-4cee-92b5-8194d35a4c8f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.158377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40adb2c0-94d3-4cee-92b5-8194d35a4c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.177072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 827/1636] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:00 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:00.196139 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:00.204846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-40adb2c0-94d3-4cee-92b5-8194d35a4c8f None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:00.206556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-40adb2c0-94d3-4cee-92b5-8194d35a4c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:00.389527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 810/1637] 10.222.0.25 () {58 vars in 1059 bytes} [Tue Sep 13 07:01:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:00.395300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950e21c-dc97-4ac9-bf59-41533630c9f8 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.395845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950e21c-dc97-4ac9-bf59-41533630c9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.396751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950e21c-dc97-4ac9-bf59-41533630c9f8 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.398724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 828/1638] 10.222.0.25 () {54 vars in 912 bytes} [Tue Sep 13 07:01:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:01:00.404343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db40bb-5cfb-4bb5-9d32-697767df8950 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.405244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db40bb-5cfb-4bb5-9d32-697767df8950 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.407327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6db40bb-5cfb-4bb5-9d32-697767df8950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.437506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d6db40bb-5cfb-4bb5-9d32-697767df8950 None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:00.439607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d6db40bb-5cfb-4bb5-9d32-697767df8950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:00.457150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950e21c-dc97-4ac9-bf59-41533630c9f8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:00.458240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 811/1639] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:01:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:00.491746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950e21c-dc97-4ac9-bf59-41533630c9f8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:00.504197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef965b6-f194-4d4a-a68e-3a44b0ace98a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:00.505074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef965b6-f194-4d4a-a68e-3a44b0ace98a service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.505530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef965b6-f194-4d4a-a68e-3a44b0ace98a service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.505813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef965b6-f194-4d4a-a68e-3a44b0ace98a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.514874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db40bb-5cfb-4bb5-9d32-697767df8950 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:00.542377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6db40bb-5cfb-4bb5-9d32-697767df8950 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:00.546294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84dc82a-2713-4827-bb06-19ebcf2741f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:00.547089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc82a-2713-4827-bb06-19ebcf2741f8 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.547339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc82a-2713-4827-bb06-19ebcf2741f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.547591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84dc82a-2713-4827-bb06-19ebcf2741f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.564477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dc82a-2713-4827-bb06-19ebcf2741f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:00.567793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84dc82a-2713-4827-bb06-19ebcf2741f8 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:00.589343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 812/1640] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:00.590884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046fc7e-0ab6-42e9-845c-33fffd68612b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.591177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046fc7e-0ab6-42e9-845c-33fffd68612b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.591458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f046fc7e-0ab6-42e9-845c-33fffd68612b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.600559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 813/1641] 10.222.0.25 () {54 vars in 912 bytes} [Tue Sep 13 07:01:00 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:01:00.603720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef965b6-f194-4d4a-a68e-3a44b0ace98a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:00.616323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3c8e7-f61c-4ad0-a2a8-677e2e3c7437 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.661809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3c8e7-f61c-4ad0-a2a8-677e2e3c7437 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.673857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef965b6-f194-4d4a-a68e-3a44b0ace98a service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:00.679806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc3c8e7-f61c-4ad0-a2a8-677e2e3c7437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.683206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 829/1642] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:00 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:00.735061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8cc3c8e7-f61c-4ad0-a2a8-677e2e3c7437 None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:00.736585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8cc3c8e7-f61c-4ad0-a2a8-677e2e3c7437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:00.787618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 814/1643] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:01:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:00.850113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef965b6-f194-4d4a-a68e-3a44b0ace98a service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:00.864751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef965b6-f194-4d4a-a68e-3a44b0ace98a service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:00.878725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a042ee-766f-4ca2-ab46-7922b53aabef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:00.883706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a042ee-766f-4ca2-ab46-7922b53aabef service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:00.886519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a042ee-766f-4ca2-ab46-7922b53aabef service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:00.886929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a042ee-766f-4ca2-ab46-7922b53aabef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:00.921119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a042ee-766f-4ca2-ab46-7922b53aabef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:00.922744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a042ee-766f-4ca2-ab46-7922b53aabef service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:00.955951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 830/1644] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:01.111726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3c8e7-f61c-4ad0-a2a8-677e2e3c7437 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:01.308744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc3c8e7-f61c-4ad0-a2a8-677e2e3c7437 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:01.319587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:01.320318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:01.320969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:01.320969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] PATH_INFO: `/v3/users/84f133a848ed4f64a2c1bd0fc53c0424` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:01.322265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84f133a848ed4f64a2c1bd0fc53c0424)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:01.323516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: WARNING py.warnings [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:01:01.323516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: warnings.warn(msg) Sep 13 07:01:01.323516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: Sep 13 07:01:01.336619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:01.599458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84f133a848ed4f64a2c1bd0fc53c0424'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:01.618320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84f133a848ed4f64a2c1bd0fc53c0424'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:01.663181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84f133a848ed4f64a2c1bd0fc53c0424'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:01.664713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84f133a848ed4f64a2c1bd0fc53c0424'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:01.668875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:01.680448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84f133a848ed4f64a2c1bd0fc53c0424'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:01.690032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 815/1645] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:01 2022] DELETE /identity/v3/users/84f133a848ed4f64a2c1bd0fc53c0424 => generated 0 bytes in 579 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:01.696262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3711399-3798-43d9-9a99-735a97fdf890 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:01.696767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3711399-3798-43d9-9a99-735a97fdf890 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:01.696921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3711399-3798-43d9-9a99-735a97fdf890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:01.733454 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d3711399-3798-43d9-9a99-735a97fdf890 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:01.734843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3711399-3798-43d9-9a99-735a97fdf890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:01.846672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 831/1646] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:01.885693 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:01.887156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:02.043707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade2aedc-0f75-4116-8cb7-56403dc209d8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:02.055295 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a609be6-42a4-4530-be89-28bd6048f142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:02.056299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a609be6-42a4-4530-be89-28bd6048f142 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:02.056577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a609be6-42a4-4530-be89-28bd6048f142 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:02.056845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a609be6-42a4-4530-be89-28bd6048f142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:02.075836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a609be6-42a4-4530-be89-28bd6048f142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:02.077306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a609be6-42a4-4530-be89-28bd6048f142 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:02.107278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 816/1647] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:04.655091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3711399-3798-43d9-9a99-735a97fdf890 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:04.656555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3711399-3798-43d9-9a99-735a97fdf890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:04.673908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3711399-3798-43d9-9a99-735a97fdf890 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:04.681962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:04.683464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:04.684114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:04.684564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] PATH_INFO: `/v3/projects/b5ea0c5e37d54c28b948355d9c4be3f2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:04.685947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5ea0c5e37d54c28b948355d9c4be3f2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:04.687221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: WARNING py.warnings [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 13 07:01:04.687221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: warnings.warn(msg) Sep 13 07:01:04.687221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: Sep 13 07:01:04.688979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:04.789872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:04.821196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5ea0c5e37d54c28b948355d9c4be3f2', 'request_id': 'req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:04.853124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5ea0c5e37d54c28b948355d9c4be3f2', 'request_id': 'req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:04.869834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5ea0c5e37d54c28b948355d9c4be3f2', 'request_id': 'req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:04.894513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] The token cache is being invalidate because project b5ea0c5e37d54c28b948355d9c4be3f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:04.894811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:04.902573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4c241ef8-35b9-42d1-891f-4b9f0ce2f9da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:04.907558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 832/1648] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:04 2022] DELETE /identity/v3/projects/b5ea0c5e37d54c28b948355d9c4be3f2 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:04.909262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a1d23-ec11-441d-ba42-48828c384911 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:04.909810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a1d23-ec11-441d-ba42-48828c384911 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:04.909810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929a1d23-ec11-441d-ba42-48828c384911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:04.946689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-929a1d23-ec11-441d-ba42-48828c384911 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:04.947987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-929a1d23-ec11-441d-ba42-48828c384911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:05.025328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 817/1649] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:05.037082 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23b3a1-6109-4901-9c1e-82c915447ac1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:05.037496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23b3a1-6109-4901-9c1e-82c915447ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:05.037886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f23b3a1-6109-4901-9c1e-82c915447ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:05.067478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8f23b3a1-6109-4901-9c1e-82c915447ac1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:05.069101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8f23b3a1-6109-4901-9c1e-82c915447ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:05.083557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 833/1650] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:05.088632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a1d23-ec11-441d-ba42-48828c384911 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:05.102022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929a1d23-ec11-441d-ba42-48828c384911 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:05.109438 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c91796-a992-4e27-b525-bf1233420296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:05.110309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c91796-a992-4e27-b525-bf1233420296 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:05.110597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c91796-a992-4e27-b525-bf1233420296 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:05.110998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c91796-a992-4e27-b525-bf1233420296 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:05.111823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c91796-a992-4e27-b525-bf1233420296 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:05.112623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c91796-a992-4e27-b525-bf1233420296 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:05.115758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c91796-a992-4e27-b525-bf1233420296 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:05.136853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 818/1651] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:01:05.139194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23b3a1-6109-4901-9c1e-82c915447ac1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:05.153568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f23b3a1-6109-4901-9c1e-82c915447ac1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:05.160245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d9104-e9da-4ff4-acad-678ccac89ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:05.160890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d9104-e9da-4ff4-acad-678ccac89ec0 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:05.161169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d9104-e9da-4ff4-acad-678ccac89ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:05.161602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d9104-e9da-4ff4-acad-678ccac89ec0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:05.162169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d9104-e9da-4ff4-acad-678ccac89ec0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:05.164162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d9104-e9da-4ff4-acad-678ccac89ec0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:05.203895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 834/1652] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:05 2022] POST /identity/v3/projects => generated 369 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:05.208454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c91796-a992-4e27-b525-bf1233420296 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:05.225312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c91796-a992-4e27-b525-bf1233420296 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:05.346043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac992d-9d4e-43d6-89bf-0ebcd251e3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:05.347077 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac992d-9d4e-43d6-89bf-0ebcd251e3c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:05.348415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac992d-9d4e-43d6-89bf-0ebcd251e3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:05.349161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac992d-9d4e-43d6-89bf-0ebcd251e3c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:05.350330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac992d-9d4e-43d6-89bf-0ebcd251e3c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:05.353304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ac992d-9d4e-43d6-89bf-0ebcd251e3c8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:05.399342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 819/1653] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:05 2022] POST /identity/v3/users => generated 285 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:05.400159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d9104-e9da-4ff4-acad-678ccac89ec0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:05.437836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d9104-e9da-4ff4-acad-678ccac89ec0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:05.458847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b91b4-7b96-46e9-8210-2c533311af63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:05.459772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b91b4-7b96-46e9-8210-2c533311af63 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:05.460246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b91b4-7b96-46e9-8210-2c533311af63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:05.460764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8b91b4-7b96-46e9-8210-2c533311af63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:05.461675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b91b4-7b96-46e9-8210-2c533311af63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:05.464031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8b91b4-7b96-46e9-8210-2c533311af63 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:05.482132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 835/1654] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:05.485926 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac992d-9d4e-43d6-89bf-0ebcd251e3c8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:05.501303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac992d-9d4e-43d6-89bf-0ebcd251e3c8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:05.517128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a73e6bb-637c-458f-8356-ec3185fb7b1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:05.519674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73e6bb-637c-458f-8356-ec3185fb7b1a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:05.519978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73e6bb-637c-458f-8356-ec3185fb7b1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:05.520250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a73e6bb-637c-458f-8356-ec3185fb7b1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:05.520919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73e6bb-637c-458f-8356-ec3185fb7b1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:05.527942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a73e6bb-637c-458f-8356-ec3185fb7b1a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:05.545604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:05.558588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 820/1655] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:05.559935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b91b4-7b96-46e9-8210-2c533311af63 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:05.582865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8b91b4-7b96-46e9-8210-2c533311af63 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:05.605509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef3878-a4e9-4150-89e2-6abc69060300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:05.606911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef3878-a4e9-4150-89e2-6abc69060300 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:05.607488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef3878-a4e9-4150-89e2-6abc69060300 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:05.608046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef3878-a4e9-4150-89e2-6abc69060300 admin admin] PATH_INFO: `/v3/projects/25921d874fb04156b4012fa369987838/users/7f5ee6d7b64943019e69ea4091831e5b/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:05.643755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef3878-a4e9-4150-89e2-6abc69060300 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25921d874fb04156b4012fa369987838, user_id=7f5ee6d7b64943019e69ea4091831e5b, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:05.652855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef3878-a4e9-4150-89e2-6abc69060300 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:05.688930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 836/1656] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:05 2022] PUT /identity/v3/projects/25921d874fb04156b4012fa369987838/users/7f5ee6d7b64943019e69ea4091831e5b/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:05.701598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94b738-4182-46c3-b53b-79a65d798477 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:05.701911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94b738-4182-46c3-b53b-79a65d798477 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:05.703292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b94b738-4182-46c3-b53b-79a65d798477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:05.774236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0b94b738-4182-46c3-b53b-79a65d798477 None None] MFA Rules not processed for user `7f5ee6d7b64943019e69ea4091831e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:05.777812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0b94b738-4182-46c3-b53b-79a65d798477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:05.983115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 821/1657] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:05.986444 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef3878-a4e9-4150-89e2-6abc69060300 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:06.138221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef3878-a4e9-4150-89e2-6abc69060300 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:06.148191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44b61f-6ef3-495a-8a96-ff847510f4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:06.149065 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b61f-6ef3-495a-8a96-ff847510f4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:06.149480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b61f-6ef3-495a-8a96-ff847510f4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:06.149949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44b61f-6ef3-495a-8a96-ff847510f4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:06.151813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44b61f-6ef3-495a-8a96-ff847510f4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:06.153124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b826fc-7985-45e5-94cb-3424f7a59478 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:06.153421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b826fc-7985-45e5-94cb-3424f7a59478 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:06.153694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b826fc-7985-45e5-94cb-3424f7a59478 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:06.160432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44b61f-6ef3-495a-8a96-ff847510f4f5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:06.201044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 837/1658] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:05 2022] POST /identity/v3/projects => generated 371 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:06.205442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44b61f-6ef3-495a-8a96-ff847510f4f5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:06.247362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a2b826fc-7985-45e5-94cb-3424f7a59478 None None] MFA Rules not processed for user `5a0fd7347cad41bc80f427f872cf78fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:06.248797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a2b826fc-7985-45e5-94cb-3424f7a59478 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:06.273093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:06.312096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44b61f-6ef3-495a-8a96-ff847510f4f5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:06.337221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3510b18-9321-429d-96e0-383399549576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:06.337891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3510b18-9321-429d-96e0-383399549576 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:06.338200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3510b18-9321-429d-96e0-383399549576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:06.338488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3510b18-9321-429d-96e0-383399549576 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:06.339141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3510b18-9321-429d-96e0-383399549576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:06.341119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3510b18-9321-429d-96e0-383399549576 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:06.383085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 838/1659] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:06 2022] POST /identity/v3/users => generated 292 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:01:06.384681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f5a83-f2f1-48f0-939c-0bfeeb268aec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:06.385979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f5a83-f2f1-48f0-939c-0bfeeb268aec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:06.392355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4f5a83-f2f1-48f0-939c-0bfeeb268aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:06.417430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 822/1660] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:06 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:06.418140 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b826fc-7985-45e5-94cb-3424f7a59478 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:06.436883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b826fc-7985-45e5-94cb-3424f7a59478 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:06.463529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a719a27-dbbc-4ad8-9953-886e5aa0d859 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:06.470435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a719a27-dbbc-4ad8-9953-886e5aa0d859 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:06.471071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a719a27-dbbc-4ad8-9953-886e5aa0d859 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:06.473667 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a719a27-dbbc-4ad8-9953-886e5aa0d859 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:06.474275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-dc4f5a83-f2f1-48f0-939c-0bfeeb268aec None None] MFA Rules not processed for user `0d8d0ff010c04d65b36ebe65c1cbbbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:06.474989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a719a27-dbbc-4ad8-9953-886e5aa0d859 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:06.475458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dc4f5a83-f2f1-48f0-939c-0bfeeb268aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:06.477480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a719a27-dbbc-4ad8-9953-886e5aa0d859 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:06.496374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 823/1661] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:06.497544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a719a27-dbbc-4ad8-9953-886e5aa0d859 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:06.501541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9a719a27-dbbc-4ad8-9953-886e5aa0d859 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:06.623346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 839/1662] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:06.623973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f5a83-f2f1-48f0-939c-0bfeeb268aec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:06.659223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4f5a83-f2f1-48f0-939c-0bfeeb268aec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:06.678819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409c1f6-50db-4419-b6ae-6b71319ca784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:06.679694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409c1f6-50db-4419-b6ae-6b71319ca784 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:06.681747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409c1f6-50db-4419-b6ae-6b71319ca784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:06.682040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409c1f6-50db-4419-b6ae-6b71319ca784 admin admin] PATH_INFO: `/v3/projects/da621f8b30794715966ef2bc4ead7c2e/users/8fc7864aced24264be451243b69afa8f/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:06.701179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409c1f6-50db-4419-b6ae-6b71319ca784 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da621f8b30794715966ef2bc4ead7c2e, user_id=8fc7864aced24264be451243b69afa8f, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:06.705436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409c1f6-50db-4419-b6ae-6b71319ca784 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:06.726061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a719a27-dbbc-4ad8-9953-886e5aa0d859 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:06.727312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 840/1663] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:06 2022] PUT /identity/v3/projects/da621f8b30794715966ef2bc4ead7c2e/users/8fc7864aced24264be451243b69afa8f/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:06.728938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14ebdb-3eec-48c5-8bfa-85d55ec984e8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:06.729305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14ebdb-3eec-48c5-8bfa-85d55ec984e8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:06.729588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14ebdb-3eec-48c5-8bfa-85d55ec984e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:06.738567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:06.756214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4d14ebdb-3eec-48c5-8bfa-85d55ec984e8 None None] MFA Rules not processed for user `0d8d0ff010c04d65b36ebe65c1cbbbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:06.758355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4d14ebdb-3eec-48c5-8bfa-85d55ec984e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:06.838191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 841/1664] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:06 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:06.838743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14ebdb-3eec-48c5-8bfa-85d55ec984e8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:06.894688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d2943-e134-4da3-b609-561739511885 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:06.895607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2943-e134-4da3-b609-561739511885 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:06.895893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2943-e134-4da3-b609-561739511885 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:06.896170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14d2943-e134-4da3-b609-561739511885 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:06.927523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:07.000763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14ebdb-3eec-48c5-8bfa-85d55ec984e8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:07.017893 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885bd5f-dc11-4991-8b76-bdefad4dae84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:07.021507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885bd5f-dc11-4991-8b76-bdefad4dae84 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:07.023483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885bd5f-dc11-4991-8b76-bdefad4dae84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:07.023804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1885bd5f-dc11-4991-8b76-bdefad4dae84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:07.026702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885bd5f-dc11-4991-8b76-bdefad4dae84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:07.038748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1885bd5f-dc11-4991-8b76-bdefad4dae84 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:07.068185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d2943-e134-4da3-b609-561739511885 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:07.071491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14d2943-e134-4da3-b609-561739511885 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:07.074566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 842/1665] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:06 2022] GET /identity/v3/roles => generated 1420 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:07.079081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885bd5f-dc11-4991-8b76-bdefad4dae84 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:07.084666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1885bd5f-dc11-4991-8b76-bdefad4dae84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:07.099972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 824/1666] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:06 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:07.101101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d2943-e134-4da3-b609-561739511885 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:07.139324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14d2943-e134-4da3-b609-561739511885 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:07.154765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1885bd5f-dc11-4991-8b76-bdefad4dae84 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:07.174549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a570071d-9622-4332-be90-25afa5bdb26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:07.176490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570071d-9622-4332-be90-25afa5bdb26a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:07.176783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570071d-9622-4332-be90-25afa5bdb26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:07.177062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a570071d-9622-4332-be90-25afa5bdb26a admin admin] PATH_INFO: `/v3/projects/da621f8b30794715966ef2bc4ead7c2e/users/8fc7864aced24264be451243b69afa8f/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:07.177265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747574-51ad-4bb0-8999-8aad1e1daf18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:07.178179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747574-51ad-4bb0-8999-8aad1e1daf18 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:07.178708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747574-51ad-4bb0-8999-8aad1e1daf18 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:07.179001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba747574-51ad-4bb0-8999-8aad1e1daf18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:07.188194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570071d-9622-4332-be90-25afa5bdb26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=da621f8b30794715966ef2bc4ead7c2e, user_id=8fc7864aced24264be451243b69afa8f, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:07.189936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba747574-51ad-4bb0-8999-8aad1e1daf18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:07.191826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba747574-51ad-4bb0-8999-8aad1e1daf18 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:07.194635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a570071d-9622-4332-be90-25afa5bdb26a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:07.219972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 843/1667] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:07 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:07.238942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747574-51ad-4bb0-8999-8aad1e1daf18 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:07.249116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ba747574-51ad-4bb0-8999-8aad1e1daf18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:07.269951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 825/1668] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:07 2022] PUT /identity/v3/projects/da621f8b30794715966ef2bc4ead7c2e/users/8fc7864aced24264be451243b69afa8f/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:07.281205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d39e9e-5c19-4a78-83be-831f111306ec None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:07.281633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d39e9e-5c19-4a78-83be-831f111306ec None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:07.282058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d39e9e-5c19-4a78-83be-831f111306ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:07.364186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-08d39e9e-5c19-4a78-83be-831f111306ec None None] MFA Rules not processed for user `8fc7864aced24264be451243b69afa8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:07.368118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-08d39e9e-5c19-4a78-83be-831f111306ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:07.468803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba747574-51ad-4bb0-8999-8aad1e1daf18 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:07.613275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 826/1669] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:07 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 329 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:07.659789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8038719a-e132-434d-8272-64d793c05cdf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:07.660882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038719a-e132-434d-8272-64d793c05cdf service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:07.661168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038719a-e132-434d-8272-64d793c05cdf service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:07.661442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8038719a-e132-434d-8272-64d793c05cdf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:07.667255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8038719a-e132-434d-8272-64d793c05cdf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:07.860572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038719a-e132-434d-8272-64d793c05cdf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:07.862207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8038719a-e132-434d-8272-64d793c05cdf service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:07.887060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 844/1670] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 660 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:08.050008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d39e9e-5c19-4a78-83be-831f111306ec None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:08.051871 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-08d39e9e-5c19-4a78-83be-831f111306ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:08.091678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d39e9e-5c19-4a78-83be-831f111306ec None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:08.101292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39936d45-c260-4ac8-b492-ad856e54ea34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:08.104448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39936d45-c260-4ac8-b492-ad856e54ea34 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:08.108094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39936d45-c260-4ac8-b492-ad856e54ea34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:08.108496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39936d45-c260-4ac8-b492-ad856e54ea34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:08.142605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39936d45-c260-4ac8-b492-ad856e54ea34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:08.144230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39936d45-c260-4ac8-b492-ad856e54ea34 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:08.158508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 827/1671] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:08 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:08.400392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9a84b-8809-440b-aaef-1e34e7d3c4cc None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:08.401234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9a84b-8809-440b-aaef-1e34e7d3c4cc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:08.401543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c9a84b-8809-440b-aaef-1e34e7d3c4cc None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:08.412088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 845/1672] 10.222.0.25 () {54 vars in 912 bytes} [Tue Sep 13 07:01:08 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:01:08.419010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19471c9-8456-44aa-9f21-ea1a26dc394b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:08.419519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19471c9-8456-44aa-9f21-ea1a26dc394b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:08.420010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19471c9-8456-44aa-9f21-ea1a26dc394b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:08.427458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:08.508517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f19471c9-8456-44aa-9f21-ea1a26dc394b None None] MFA Rules not processed for user `f94aa8af227a42a08d8a0e4b7f02b85a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:08.524508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f19471c9-8456-44aa-9f21-ea1a26dc394b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:08.664768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 828/1673] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:01:08 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:08.689383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9a84b-8809-440b-aaef-1e34e7d3c4cc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:08.692299 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f2c9a84b-8809-440b-aaef-1e34e7d3c4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:08.737538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c9a84b-8809-440b-aaef-1e34e7d3c4cc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:08.750693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48162eea-4297-4494-a617-f8777b390217 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:08.751423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48162eea-4297-4494-a617-f8777b390217 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:08.751899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48162eea-4297-4494-a617-f8777b390217 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:08.752306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48162eea-4297-4494-a617-f8777b390217 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:08.780463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48162eea-4297-4494-a617-f8777b390217 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:08.782018 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48162eea-4297-4494-a617-f8777b390217 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:08.811483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 846/1674] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:09.991711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19471c9-8456-44aa-9f21-ea1a26dc394b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:09.993274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f19471c9-8456-44aa-9f21-ea1a26dc394b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:10.210646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19471c9-8456-44aa-9f21-ea1a26dc394b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:10.212655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48162eea-4297-4494-a617-f8777b390217 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:10.221928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46e44-6ef9-4af4-ae98-aa8aa7f16b1b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:10.222804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46e44-6ef9-4af4-ae98-aa8aa7f16b1b service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:10.223441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46e44-6ef9-4af4-ae98-aa8aa7f16b1b service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:10.223955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46e44-6ef9-4af4-ae98-aa8aa7f16b1b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:10.232967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-adb46e44-6ef9-4af4-ae98-aa8aa7f16b1b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:10.264461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46e44-6ef9-4af4-ae98-aa8aa7f16b1b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:10.266928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46e44-6ef9-4af4-ae98-aa8aa7f16b1b service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:10.281213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 829/1675] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:10.289660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19739bf5-bdd9-4a41-b22c-33d469bb5a2c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:10.289957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19739bf5-bdd9-4a41-b22c-33d469bb5a2c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:10.290289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19739bf5-bdd9-4a41-b22c-33d469bb5a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:10.295256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:10.313756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-19739bf5-bdd9-4a41-b22c-33d469bb5a2c None None] MFA Rules not processed for user `0d8d0ff010c04d65b36ebe65c1cbbbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:10.315196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-19739bf5-bdd9-4a41-b22c-33d469bb5a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:10.402925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 830/1676] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:10 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:10.488227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48162eea-4297-4494-a617-f8777b390217 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:10.506543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:10.507423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:10.507552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:10.507750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] PATH_INFO: `/v3/users/7f5ee6d7b64943019e69ea4091831e5b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:10.515682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f5ee6d7b64943019e69ea4091831e5b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:10.516218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:10.720282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f5ee6d7b64943019e69ea4091831e5b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:10.720580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f5ee6d7b64943019e69ea4091831e5b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:10.741068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f5ee6d7b64943019e69ea4091831e5b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:10.776244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f5ee6d7b64943019e69ea4091831e5b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:10.779226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f5ee6d7b64943019e69ea4091831e5b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:10.824612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 847/1677] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:10 2022] DELETE /identity/v3/users/7f5ee6d7b64943019e69ea4091831e5b => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:10.829645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caf24e-566c-4eb7-ad5b-df647f3d5270 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:10.829953 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caf24e-566c-4eb7-ad5b-df647f3d5270 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:10.830277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97caf24e-566c-4eb7-ad5b-df647f3d5270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:10.870220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-97caf24e-566c-4eb7-ad5b-df647f3d5270 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:10.873830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-97caf24e-566c-4eb7-ad5b-df647f3d5270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:11.006363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:11.084547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 831/1678] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:11.197765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:11.199672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:11.356970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97133ba-ac0e-4026-a45c-acdea2f93ec2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:11.376123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eb428-910b-46ea-82e4-867398907343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:11.376783 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eb428-910b-46ea-82e4-867398907343 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:11.377066 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eb428-910b-46ea-82e4-867398907343 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:11.377348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488eb428-910b-46ea-82e4-867398907343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:11.404822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488eb428-910b-46ea-82e4-867398907343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:11.406339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488eb428-910b-46ea-82e4-867398907343 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:11.452732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 848/1679] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:12.144491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda182c-d743-48cb-a2c2-b7dfc75f1623 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:12.145497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda182c-d743-48cb-a2c2-b7dfc75f1623 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:12.146845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbda182c-d743-48cb-a2c2-b7dfc75f1623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:12.265227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cbda182c-d743-48cb-a2c2-b7dfc75f1623 None None] MFA Rules not processed for user `ecf2dfd9bbe54b3085e9638caf8ad044`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:12.266528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cbda182c-d743-48cb-a2c2-b7dfc75f1623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:12.472656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 832/1680] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:12 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:12.479901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eb428-910b-46ea-82e4-867398907343 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:12.485674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-488eb428-910b-46ea-82e4-867398907343 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:12.604414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:12.689425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488eb428-910b-46ea-82e4-867398907343 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:12.716058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639dfe2-9975-489f-a3e3-5fe9dea6bfe5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:12.716720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639dfe2-9975-489f-a3e3-5fe9dea6bfe5 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:12.717006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639dfe2-9975-489f-a3e3-5fe9dea6bfe5 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:12.720205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b639dfe2-9975-489f-a3e3-5fe9dea6bfe5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:12.746220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639dfe2-9975-489f-a3e3-5fe9dea6bfe5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:12.755199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b639dfe2-9975-489f-a3e3-5fe9dea6bfe5 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:12.782289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 849/1681] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:12 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:13.290873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda182c-d743-48cb-a2c2-b7dfc75f1623 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:13.293655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cbda182c-d743-48cb-a2c2-b7dfc75f1623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:13.325057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbda182c-d743-48cb-a2c2-b7dfc75f1623 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:13.330418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:13.331235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:13.331776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:13.332059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] PATH_INFO: `/v3/projects/25921d874fb04156b4012fa369987838` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:13.333280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=25921d874fb04156b4012fa369987838)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:13.350568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:13.468016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25921d874fb04156b4012fa369987838', 'request_id': 'req-61afcdc4-9412-4061-b392-cb87279b5cd7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.480377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25921d874fb04156b4012fa369987838', 'request_id': 'req-61afcdc4-9412-4061-b392-cb87279b5cd7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.491502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25921d874fb04156b4012fa369987838', 'request_id': 'req-61afcdc4-9412-4061-b392-cb87279b5cd7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.519886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] The token cache is being invalidate because project 25921d874fb04156b4012fa369987838 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:13.526578 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.527541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-61afcdc4-9412-4061-b392-cb87279b5cd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.532498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 833/1682] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:13 2022] DELETE /identity/v3/projects/25921d874fb04156b4012fa369987838 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:13.534480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639dfe2-9975-489f-a3e3-5fe9dea6bfe5 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:13.553696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b639dfe2-9975-489f-a3e3-5fe9dea6bfe5 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:13.757591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:13.758122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:13.758418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:13.758723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] PATH_INFO: `/v3/users/8fc7864aced24264be451243b69afa8f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:13.765138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc7864aced24264be451243b69afa8f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:13.769061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:13.845221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc7864aced24264be451243b69afa8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.847227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc7864aced24264be451243b69afa8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.867927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc7864aced24264be451243b69afa8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.893816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc7864aced24264be451243b69afa8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.895386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-25af4adb-d8f7-431c-8825-9f7104e9b73d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc7864aced24264be451243b69afa8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:13.949866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 850/1683] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:13 2022] DELETE /identity/v3/users/8fc7864aced24264be451243b69afa8f => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:14.298737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a197b-1f76-4aee-84ee-540a40d3ede5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:14.299457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a197b-1f76-4aee-84ee-540a40d3ede5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:14.299566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277a197b-1f76-4aee-84ee-540a40d3ede5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:14.341437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-277a197b-1f76-4aee-84ee-540a40d3ede5 None None] MFA Rules not processed for user `5a0fd7347cad41bc80f427f872cf78fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:14.343408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-277a197b-1f76-4aee-84ee-540a40d3ede5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:14.454270 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:14.509087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 834/1684] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:14.920089 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdc6da-056f-41aa-a11f-72268c58c65e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:14.921130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdc6da-056f-41aa-a11f-72268c58c65e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:14.921130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bdc6da-056f-41aa-a11f-72268c58c65e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:15.001608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a0bdc6da-056f-41aa-a11f-72268c58c65e None None] MFA Rules not processed for user `23088396d0ca47c5929690820a3f7e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:15.003109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a0bdc6da-056f-41aa-a11f-72268c58c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:15.120325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:15.199275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 851/1685] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:15.210891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470f9db-100d-4714-8795-a4502199ba2f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:15.214051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470f9db-100d-4714-8795-a4502199ba2f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:15.214664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7470f9db-100d-4714-8795-a4502199ba2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:15.303927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7470f9db-100d-4714-8795-a4502199ba2f None None] MFA Rules not processed for user `23088396d0ca47c5929690820a3f7e29`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:15.304952 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7470f9db-100d-4714-8795-a4502199ba2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:15.332661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 835/1686] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:15 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:15.347578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdc6da-056f-41aa-a11f-72268c58c65e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:15.349481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a0bdc6da-056f-41aa-a11f-72268c58c65e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:15.358214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470f9db-100d-4714-8795-a4502199ba2f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:15.361427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7470f9db-100d-4714-8795-a4502199ba2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:15.664572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7470f9db-100d-4714-8795-a4502199ba2f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:15.681769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:15.682480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:15.682764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:15.683040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] PATH_INFO: `/v3/projects/da621f8b30794715966ef2bc4ead7c2e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:15.689214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da621f8b30794715966ef2bc4ead7c2e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:15.703765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:15.765841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bdc6da-056f-41aa-a11f-72268c58c65e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:15.780969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e7808e-955a-48b7-a3b4-e0e12d038142 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:15.782287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7808e-955a-48b7-a3b4-e0e12d038142 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:15.783519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7808e-955a-48b7-a3b4-e0e12d038142 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:15.784530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e7808e-955a-48b7-a3b4-e0e12d038142 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:15.861191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7808e-955a-48b7-a3b4-e0e12d038142 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:15.876342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da621f8b30794715966ef2bc4ead7c2e', 'request_id': 'req-2fc51907-065a-4b38-9808-31684696d836'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:15.886683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e7808e-955a-48b7-a3b4-e0e12d038142 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:15.893093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da621f8b30794715966ef2bc4ead7c2e', 'request_id': 'req-2fc51907-065a-4b38-9808-31684696d836'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:15.909658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da621f8b30794715966ef2bc4ead7c2e', 'request_id': 'req-2fc51907-065a-4b38-9808-31684696d836'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:15.953036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] The token cache is being invalidate because project da621f8b30794715966ef2bc4ead7c2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:15.953334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:15.956021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2fc51907-065a-4b38-9808-31684696d836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:15.959758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 852/1687] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:15 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:15.960711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 836/1688] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:15 2022] DELETE /identity/v3/projects/da621f8b30794715966ef2bc4ead7c2e => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:15.974168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37bd52-0842-456f-bd53-12e4f90d8444 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:15.974653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37bd52-0842-456f-bd53-12e4f90d8444 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:15.975381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37bd52-0842-456f-bd53-12e4f90d8444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:16.004466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fb37bd52-0842-456f-bd53-12e4f90d8444 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:16.006014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fb37bd52-0842-456f-bd53-12e4f90d8444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:16.249151 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 853/1689] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:16.257333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51ee57-5909-4174-ba1b-df4abace4e63 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:16.257775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51ee57-5909-4174-ba1b-df4abace4e63 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:16.258190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51ee57-5909-4174-ba1b-df4abace4e63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:16.319007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5a51ee57-5909-4174-ba1b-df4abace4e63 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:16.323828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5a51ee57-5909-4174-ba1b-df4abace4e63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:16.364430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 837/1690] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:16.369173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37bd52-0842-456f-bd53-12e4f90d8444 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:16.381896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37bd52-0842-456f-bd53-12e4f90d8444 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:16.396673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c90ded-1d66-4efb-b09a-3f7d27e5fbc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:16.397317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c90ded-1d66-4efb-b09a-3f7d27e5fbc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:16.397594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c90ded-1d66-4efb-b09a-3f7d27e5fbc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:16.397863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c90ded-1d66-4efb-b09a-3f7d27e5fbc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:16.405247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c90ded-1d66-4efb-b09a-3f7d27e5fbc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:16.405568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c90ded-1d66-4efb-b09a-3f7d27e5fbc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:16.408640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c90ded-1d66-4efb-b09a-3f7d27e5fbc4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:16.425514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd62720-7745-4c3c-8e7a-f8e0b2d6b284 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:16.425797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd62720-7745-4c3c-8e7a-f8e0b2d6b284 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:16.426097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd62720-7745-4c3c-8e7a-f8e0b2d6b284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:16.452313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 854/1691] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:16.456265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c90ded-1d66-4efb-b09a-3f7d27e5fbc4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:16.487199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c90ded-1d66-4efb-b09a-3f7d27e5fbc4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:16.499374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8ed9c-7ea4-4d08-bea2-4a4ce2e7b012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:16.500028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8ed9c-7ea4-4d08-bea2-4a4ce2e7b012 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:16.500311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8ed9c-7ea4-4d08-bea2-4a4ce2e7b012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:16.500630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8ed9c-7ea4-4d08-bea2-4a4ce2e7b012 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:16.503254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8ed9c-7ea4-4d08-bea2-4a4ce2e7b012 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:16.508977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8ed9c-7ea4-4d08-bea2-4a4ce2e7b012 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:16.576036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4dd62720-7745-4c3c-8e7a-f8e0b2d6b284 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:16.581920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4dd62720-7745-4c3c-8e7a-f8e0b2d6b284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:16.598494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 855/1692] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:16 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:16.602559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8ed9c-7ea4-4d08-bea2-4a4ce2e7b012 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:16.676990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:16.764414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:16.820280 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8ed9c-7ea4-4d08-bea2-4a4ce2e7b012 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:16.851939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0b5c9-5169-4419-a1d5-1c10fbc0f047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:16.852842 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0b5c9-5169-4419-a1d5-1c10fbc0f047 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:16.856192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0b5c9-5169-4419-a1d5-1c10fbc0f047 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:16.857526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca0b5c9-5169-4419-a1d5-1c10fbc0f047 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:16.862986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0b5c9-5169-4419-a1d5-1c10fbc0f047 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:16.868712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 838/1693] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:01:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 13 07:01:16.872768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca0b5c9-5169-4419-a1d5-1c10fbc0f047 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:16.936925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 856/1694] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:16 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:01:16.943374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd62720-7745-4c3c-8e7a-f8e0b2d6b284 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:16.961766 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd62720-7745-4c3c-8e7a-f8e0b2d6b284 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:16.973945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680cbc7-1f1e-4fec-9d44-41f147ff982a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:16.974607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680cbc7-1f1e-4fec-9d44-41f147ff982a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:16.976986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680cbc7-1f1e-4fec-9d44-41f147ff982a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:16.977310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3680cbc7-1f1e-4fec-9d44-41f147ff982a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:16.977987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680cbc7-1f1e-4fec-9d44-41f147ff982a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:16.985227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3680cbc7-1f1e-4fec-9d44-41f147ff982a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:17.012631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 839/1695] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:16 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:17.016894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0b5c9-5169-4419-a1d5-1c10fbc0f047 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:17.035600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca0b5c9-5169-4419-a1d5-1c10fbc0f047 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:17.050070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5711a1-7457-4cc8-9992-e3334a25da8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:17.057947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5711a1-7457-4cc8-9992-e3334a25da8e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:17.060437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5711a1-7457-4cc8-9992-e3334a25da8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:17.061230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5711a1-7457-4cc8-9992-e3334a25da8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:17.064413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5711a1-7457-4cc8-9992-e3334a25da8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:17.068907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5711a1-7457-4cc8-9992-e3334a25da8e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:17.135140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 857/1696] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:17 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:17.137812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680cbc7-1f1e-4fec-9d44-41f147ff982a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:17.176531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3680cbc7-1f1e-4fec-9d44-41f147ff982a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:17.206565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6f7432-978e-486e-8bfa-38d68d017867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:17.209492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f7432-978e-486e-8bfa-38d68d017867 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:17.211441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f7432-978e-486e-8bfa-38d68d017867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:17.212078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6f7432-978e-486e-8bfa-38d68d017867 admin admin] PATH_INFO: `/v3/projects/09e7b589f6734adc8ac14644b6888355/users/aad48d2bb116464b92350939891956f0/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:17.222532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5711a1-7457-4cc8-9992-e3334a25da8e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:17.224978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ab5711a1-7457-4cc8-9992-e3334a25da8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:17.269215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f7432-978e-486e-8bfa-38d68d017867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09e7b589f6734adc8ac14644b6888355, user_id=aad48d2bb116464b92350939891956f0, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:17.273573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6f7432-978e-486e-8bfa-38d68d017867 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:17.318358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 840/1697] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:17 2022] PUT /identity/v3/projects/09e7b589f6734adc8ac14644b6888355/users/aad48d2bb116464b92350939891956f0/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:17.344581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcf324-0ee1-4ab5-a6b6-82fda39a0ea5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:17.345599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcf324-0ee1-4ab5-a6b6-82fda39a0ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:17.346168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efcf324-0ee1-4ab5-a6b6-82fda39a0ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:17.432001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5efcf324-0ee1-4ab5-a6b6-82fda39a0ea5 None None] MFA Rules not processed for user `aad48d2bb116464b92350939891956f0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:17.435909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5efcf324-0ee1-4ab5-a6b6-82fda39a0ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:17.503679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5711a1-7457-4cc8-9992-e3334a25da8e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:17.606473 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 841/1698] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:17 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:17.616330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcf324-0ee1-4ab5-a6b6-82fda39a0ea5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:17.714942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569e7808-449f-4160-8491-13d986ebaf60 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:17.715915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e7808-449f-4160-8491-13d986ebaf60 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:17.716273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e7808-449f-4160-8491-13d986ebaf60 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:17.716626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569e7808-449f-4160-8491-13d986ebaf60 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:17.770782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:17.806873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efcf324-0ee1-4ab5-a6b6-82fda39a0ea5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:17.817554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e7808-449f-4160-8491-13d986ebaf60 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:17.818035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b459a4-83c5-4f76-8efa-9bb4eed92aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:17.818998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b459a4-83c5-4f76-8efa-9bb4eed92aea admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:17.819112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b459a4-83c5-4f76-8efa-9bb4eed92aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:17.819472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b459a4-83c5-4f76-8efa-9bb4eed92aea admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:17.820240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569e7808-449f-4160-8491-13d986ebaf60 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:17.820344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b459a4-83c5-4f76-8efa-9bb4eed92aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:17.822629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b459a4-83c5-4f76-8efa-9bb4eed92aea admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:17.832434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 858/1699] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:01:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:01:17.834241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51d76-ba58-4304-8469-2dbd6de36bb3 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:17.835125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51d76-ba58-4304-8469-2dbd6de36bb3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:17.835524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e51d76-ba58-4304-8469-2dbd6de36bb3 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:17.837585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 859/1700] 10.222.0.25 () {54 vars in 912 bytes} [Tue Sep 13 07:01:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:01:17.844877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e27d8-2fe3-4bd0-998b-7c2b4ebbc706 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:17.845242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e27d8-2fe3-4bd0-998b-7c2b4ebbc706 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:17.845588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06e27d8-2fe3-4bd0-998b-7c2b4ebbc706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:17.899332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 842/1701] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:17 2022] POST /identity/v3/projects => generated 385 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:17.901631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b459a4-83c5-4f76-8efa-9bb4eed92aea admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:17.953379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d06e27d8-2fe3-4bd0-998b-7c2b4ebbc706 None None] MFA Rules not processed for user `f94aa8af227a42a08d8a0e4b7f02b85a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:17.971894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d06e27d8-2fe3-4bd0-998b-7c2b4ebbc706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:17.996412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:18.169596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b459a4-83c5-4f76-8efa-9bb4eed92aea admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:18.188304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828edc18-1b92-4153-a95a-1e6bfb9aded5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:18.190483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 860/1702] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:01:17 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:18.192950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828edc18-1b92-4153-a95a-1e6bfb9aded5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:18.195704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828edc18-1b92-4153-a95a-1e6bfb9aded5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:18.196204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828edc18-1b92-4153-a95a-1e6bfb9aded5 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:18.197112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828edc18-1b92-4153-a95a-1e6bfb9aded5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:18.203124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828edc18-1b92-4153-a95a-1e6bfb9aded5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:18.268440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e27d8-2fe3-4bd0-998b-7c2b4ebbc706 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:18.278547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 843/1703] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:17 2022] POST /identity/v3/users => generated 299 bytes in 377 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:18.284632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06e27d8-2fe3-4bd0-998b-7c2b4ebbc706 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:18.294409 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b8be0-643c-4e9b-be41-68615e7ac83e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:18.295326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b8be0-643c-4e9b-be41-68615e7ac83e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:18.296320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b8be0-643c-4e9b-be41-68615e7ac83e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:18.296806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37b8be0-643c-4e9b-be41-68615e7ac83e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:18.298401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b8be0-643c-4e9b-be41-68615e7ac83e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:18.301005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37b8be0-643c-4e9b-be41-68615e7ac83e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:18.356068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828edc18-1b92-4153-a95a-1e6bfb9aded5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:18.362623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 861/1704] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:18.386052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828edc18-1b92-4153-a95a-1e6bfb9aded5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:18.400662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e001ab-c53e-452e-820f-ecf98cac7a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:18.401755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e001ab-c53e-452e-820f-ecf98cac7a8e admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:18.402158 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e001ab-c53e-452e-820f-ecf98cac7a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:18.402323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b8be0-643c-4e9b-be41-68615e7ac83e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:18.402611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e001ab-c53e-452e-820f-ecf98cac7a8e admin admin] PATH_INFO: `/v3/projects/879272babecc4dfda0b9bffab5cb5432/users/f04adee616c041bf8a108a9465cbacae/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:18.404041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a37b8be0-643c-4e9b-be41-68615e7ac83e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:18.426579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37b8be0-643c-4e9b-be41-68615e7ac83e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:18.432556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e001ab-c53e-452e-820f-ecf98cac7a8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=879272babecc4dfda0b9bffab5cb5432, user_id=f04adee616c041bf8a108a9465cbacae, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:18.436879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e001ab-c53e-452e-820f-ecf98cac7a8e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:18.439786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12d783-1e6d-4992-82e2-22fddd9d1008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:18.440437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12d783-1e6d-4992-82e2-22fddd9d1008 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:18.440722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12d783-1e6d-4992-82e2-22fddd9d1008 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:18.441103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12d783-1e6d-4992-82e2-22fddd9d1008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:18.471449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12d783-1e6d-4992-82e2-22fddd9d1008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:18.472597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 844/1705] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:18 2022] PUT /identity/v3/projects/879272babecc4dfda0b9bffab5cb5432/users/f04adee616c041bf8a108a9465cbacae/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:18.476826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e001ab-c53e-452e-820f-ecf98cac7a8e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:18.480463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12d783-1e6d-4992-82e2-22fddd9d1008 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:18.480689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f6e001ab-c53e-452e-820f-ecf98cac7a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:18.511425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 862/1706] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:18.513154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12d783-1e6d-4992-82e2-22fddd9d1008 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:18.559339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12d783-1e6d-4992-82e2-22fddd9d1008 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:18.645812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e001ab-c53e-452e-820f-ecf98cac7a8e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:18.660031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:18.661478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:18.664490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:18.664771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] PATH_INFO: `/v3/users/639d3e75244543f6b5f69cdddc7c4a2e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:18.666768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] RBAC: Authorizing `identity:delete_user(user_id=639d3e75244543f6b5f69cdddc7c4a2e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:18.671989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:18.709409 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367572c-c4db-4b8b-bbcb-ba689ecc9955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:18.711944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367572c-c4db-4b8b-bbcb-ba689ecc9955 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:18.712228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367572c-c4db-4b8b-bbcb-ba689ecc9955 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:18.712495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367572c-c4db-4b8b-bbcb-ba689ecc9955 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:18.713297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367572c-c4db-4b8b-bbcb-ba689ecc9955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:18.715467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367572c-c4db-4b8b-bbcb-ba689ecc9955 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:18.743799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 863/1707] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:18 2022] GET /identity/v3/roles => generated 1420 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:18.749786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367572c-c4db-4b8b-bbcb-ba689ecc9955 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:18.780289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367572c-c4db-4b8b-bbcb-ba689ecc9955 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:18.791440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53045ea-5c93-4858-bbd8-d18153d25601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:18.794029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53045ea-5c93-4858-bbd8-d18153d25601 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:18.794343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53045ea-5c93-4858-bbd8-d18153d25601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:18.794623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53045ea-5c93-4858-bbd8-d18153d25601 admin admin] PATH_INFO: `/v3/projects/879272babecc4dfda0b9bffab5cb5432/users/f04adee616c041bf8a108a9465cbacae/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:18.800641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53045ea-5c93-4858-bbd8-d18153d25601 admin admin] RBAC: Authorizing `identity:create_grant(project_id=879272babecc4dfda0b9bffab5cb5432, user_id=f04adee616c041bf8a108a9465cbacae, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:18.810325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53045ea-5c93-4858-bbd8-d18153d25601 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:18.821581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '639d3e75244543f6b5f69cdddc7c4a2e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:18.837669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '639d3e75244543f6b5f69cdddc7c4a2e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:18.849199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 864/1708] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:18 2022] PUT /identity/v3/projects/879272babecc4dfda0b9bffab5cb5432/users/f04adee616c041bf8a108a9465cbacae/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:18.857027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350267c-c198-4081-b990-e2c2df4f2ec2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:18.857326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350267c-c198-4081-b990-e2c2df4f2ec2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:18.857645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350267c-c198-4081-b990-e2c2df4f2ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:18.875579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '639d3e75244543f6b5f69cdddc7c4a2e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:18.877735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '639d3e75244543f6b5f69cdddc7c4a2e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:18.891788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f300badf-72d7-4d8c-b266-56aee6537216 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '639d3e75244543f6b5f69cdddc7c4a2e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:18.903632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 845/1709] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:18 2022] DELETE /identity/v3/users/639d3e75244543f6b5f69cdddc7c4a2e => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:18.908653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc67b5-ac6c-452b-a64a-76b55e8bfe26 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:18.908954 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc67b5-ac6c-452b-a64a-76b55e8bfe26 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:18.909217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bc67b5-ac6c-452b-a64a-76b55e8bfe26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:18.934209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:18.944711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7350267c-c198-4081-b990-e2c2df4f2ec2 None None] MFA Rules not processed for user `f04adee616c041bf8a108a9465cbacae`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:18.946284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7350267c-c198-4081-b990-e2c2df4f2ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:18.952816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f8bc67b5-ac6c-452b-a64a-76b55e8bfe26 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:18.954091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f8bc67b5-ac6c-452b-a64a-76b55e8bfe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:19.016996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:19.074764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 865/1710] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:18 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:19.088134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350267c-c198-4081-b990-e2c2df4f2ec2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:19.089982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7350267c-c198-4081-b990-e2c2df4f2ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:19.108276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 846/1711] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:19.116043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc67b5-ac6c-452b-a64a-76b55e8bfe26 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:19.118225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f8bc67b5-ac6c-452b-a64a-76b55e8bfe26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:19.234035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350267c-c198-4081-b990-e2c2df4f2ec2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:19.239186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bef3d5f-8a6d-4af1-a86b-31ba173be6ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:19.239836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef3d5f-8a6d-4af1-a86b-31ba173be6ea service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:19.240116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef3d5f-8a6d-4af1-a86b-31ba173be6ea service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:19.240573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bef3d5f-8a6d-4af1-a86b-31ba173be6ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:19.243282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2bef3d5f-8a6d-4af1-a86b-31ba173be6ea service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:19.246816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bc67b5-ac6c-452b-a64a-76b55e8bfe26 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:19.256043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3041c20-479b-4878-9099-1bd0d0d67138 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:19.258461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3041c20-479b-4878-9099-1bd0d0d67138 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:19.258752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3041c20-479b-4878-9099-1bd0d0d67138 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:19.259022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3041c20-479b-4878-9099-1bd0d0d67138 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:19.273911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3041c20-479b-4878-9099-1bd0d0d67138 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:19.275746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3041c20-479b-4878-9099-1bd0d0d67138 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:19.289083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 847/1712] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:19.570412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef3d5f-8a6d-4af1-a86b-31ba173be6ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:19.573744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bef3d5f-8a6d-4af1-a86b-31ba173be6ea service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:19.613191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 866/1713] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 526 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:20.031337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3041c20-479b-4878-9099-1bd0d0d67138 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:20.032629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d3041c20-479b-4878-9099-1bd0d0d67138 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:20.059187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3041c20-479b-4878-9099-1bd0d0d67138 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:20.079967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:20.085394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:20.086383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:20.086620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] PATH_INFO: `/v3/projects/35fb3e3ada014f3aa7868201d2ce98d9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:20.091186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=35fb3e3ada014f3aa7868201d2ce98d9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:20.101038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:20.294924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '35fb3e3ada014f3aa7868201d2ce98d9', 'request_id': 'req-97e308ed-978b-47c7-91b9-c519b463c6d5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:20.318874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '35fb3e3ada014f3aa7868201d2ce98d9', 'request_id': 'req-97e308ed-978b-47c7-91b9-c519b463c6d5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:20.331280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '35fb3e3ada014f3aa7868201d2ce98d9', 'request_id': 'req-97e308ed-978b-47c7-91b9-c519b463c6d5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:20.377775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] The token cache is being invalidate because project 35fb3e3ada014f3aa7868201d2ce98d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:20.378501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:20.381317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97e308ed-978b-47c7-91b9-c519b463c6d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:20.394880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 848/1714] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:20 2022] DELETE /identity/v3/projects/35fb3e3ada014f3aa7868201d2ce98d9 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:20.400528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e73c7c-a91b-4aec-acee-ff6a42faa29a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:20.401083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e73c7c-a91b-4aec-acee-ff6a42faa29a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:20.401083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e73c7c-a91b-4aec-acee-ff6a42faa29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:20.483323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-40e73c7c-a91b-4aec-acee-ff6a42faa29a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:20.484577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-40e73c7c-a91b-4aec-acee-ff6a42faa29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:20.570655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694990d-395d-4b29-a339-11aece0370da None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:20.570936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694990d-395d-4b29-a339-11aece0370da None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:20.571214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4694990d-395d-4b29-a339-11aece0370da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:20.671178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4694990d-395d-4b29-a339-11aece0370da None None] MFA Rules not processed for user `f94aa8af227a42a08d8a0e4b7f02b85a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:20.672580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4694990d-395d-4b29-a339-11aece0370da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:20.798415 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 867/1715] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:20.800246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb82428-82cf-4ba6-9a99-229eec4ffe41 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:20.800750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb82428-82cf-4ba6-9a99-229eec4ffe41 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:20.804067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:20.811515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb82428-82cf-4ba6-9a99-229eec4ffe41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:20.890388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4eb82428-82cf-4ba6-9a99-229eec4ffe41 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:20.892465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4eb82428-82cf-4ba6-9a99-229eec4ffe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:20.920814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 849/1716] 10.222.0.23 () {58 vars in 1015 bytes} [Tue Sep 13 07:01:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:20.933400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 868/1717] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:20.937768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694990d-395d-4b29-a339-11aece0370da None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:20.950929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb82428-82cf-4ba6-9a99-229eec4ffe41 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:20.959220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4eb82428-82cf-4ba6-9a99-229eec4ffe41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:20.978771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4694990d-395d-4b29-a339-11aece0370da None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:21.016313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47046838-0e33-4aa4-a837-5eee78e32b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:21.016965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47046838-0e33-4aa4-a837-5eee78e32b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:21.017247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47046838-0e33-4aa4-a837-5eee78e32b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:21.017624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47046838-0e33-4aa4-a837-5eee78e32b06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:21.018882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47046838-0e33-4aa4-a837-5eee78e32b06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:21.018882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47046838-0e33-4aa4-a837-5eee78e32b06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:21.020853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47046838-0e33-4aa4-a837-5eee78e32b06 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:21.023745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb82428-82cf-4ba6-9a99-229eec4ffe41 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:21.051996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4a36b7-5644-4360-8f06-1ebce7828884 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:21.052770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a36b7-5644-4360-8f06-1ebce7828884 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:21.053525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a36b7-5644-4360-8f06-1ebce7828884 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:21.054061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4a36b7-5644-4360-8f06-1ebce7828884 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:21.093102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a36b7-5644-4360-8f06-1ebce7828884 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:21.096910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 850/1718] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:21.100964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47046838-0e33-4aa4-a837-5eee78e32b06 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:21.101959 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4a36b7-5644-4360-8f06-1ebce7828884 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:21.122259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47046838-0e33-4aa4-a837-5eee78e32b06 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:21.147438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:21.151790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd612c4-e5b8-4bba-9387-bba251800835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:21.152650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd612c4-e5b8-4bba-9387-bba251800835 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:21.153080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd612c4-e5b8-4bba-9387-bba251800835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:21.153505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd612c4-e5b8-4bba-9387-bba251800835 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:21.155523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd612c4-e5b8-4bba-9387-bba251800835 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:21.157654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd612c4-e5b8-4bba-9387-bba251800835 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:21.168262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 869/1719] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:21.169890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6d95d-ec7a-47eb-a50e-72226b7be993 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:21.174759 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6d95d-ec7a-47eb-a50e-72226b7be993 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:21.175897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc6d95d-ec7a-47eb-a50e-72226b7be993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:21.246350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8cc6d95d-ec7a-47eb-a50e-72226b7be993 None None] MFA Rules not processed for user `ecf2dfd9bbe54b3085e9638caf8ad044`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:21.248323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8cc6d95d-ec7a-47eb-a50e-72226b7be993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:21.296400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 851/1720] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:21 2022] POST /identity/v3/projects => generated 379 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:01:21.300499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd612c4-e5b8-4bba-9387-bba251800835 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:21.528825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 870/1721] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:21 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:21.643165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd612c4-e5b8-4bba-9387-bba251800835 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:21.696897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e32df2-02ae-4a7b-9d64-f9983fb85b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:21.705668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e32df2-02ae-4a7b-9d64-f9983fb85b69 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:21.722726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e32df2-02ae-4a7b-9d64-f9983fb85b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:21.723386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e32df2-02ae-4a7b-9d64-f9983fb85b69 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:21.724289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e32df2-02ae-4a7b-9d64-f9983fb85b69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:21.730935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e32df2-02ae-4a7b-9d64-f9983fb85b69 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:21.841750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6d95d-ec7a-47eb-a50e-72226b7be993 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:21.843400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8cc6d95d-ec7a-47eb-a50e-72226b7be993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:21.847381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 852/1722] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:21 2022] POST /identity/v3/users => generated 290 bytes in 545 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:21.865791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e32df2-02ae-4a7b-9d64-f9983fb85b69 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:21.921275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc6d95d-ec7a-47eb-a50e-72226b7be993 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:21.927513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e32df2-02ae-4a7b-9d64-f9983fb85b69 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:21.943175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101d83d-df44-4fb1-b95b-fd2b40e7522a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:21.943794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101d83d-df44-4fb1-b95b-fd2b40e7522a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:21.944073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101d83d-df44-4fb1-b95b-fd2b40e7522a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:21.944344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f101d83d-df44-4fb1-b95b-fd2b40e7522a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:21.945139 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101d83d-df44-4fb1-b95b-fd2b40e7522a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:21.946660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f101d83d-df44-4fb1-b95b-fd2b40e7522a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:21.977164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:21.983057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:21.991105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 853/1723] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:21.991269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:21.991269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] PATH_INFO: `/v3/users/3e9749655c0b48cb9841f7f1d4abd05e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:21.995837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e9749655c0b48cb9841f7f1d4abd05e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:21.999502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101d83d-df44-4fb1-b95b-fd2b40e7522a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:22.019730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:22.029991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f101d83d-df44-4fb1-b95b-fd2b40e7522a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:22.064176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3fb09-adfe-45a1-a31f-edb880630a6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:22.065134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3fb09-adfe-45a1-a31f-edb880630a6e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:22.066905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3fb09-adfe-45a1-a31f-edb880630a6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:22.067527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3fb09-adfe-45a1-a31f-edb880630a6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:22.068908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3fb09-adfe-45a1-a31f-edb880630a6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:22.070955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3fb09-adfe-45a1-a31f-edb880630a6e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:22.128125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 854/1724] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:21 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:22.140113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3fb09-adfe-45a1-a31f-edb880630a6e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:22.165373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3fb09-adfe-45a1-a31f-edb880630a6e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:22.204714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5969cd44-d93d-4321-9b4e-46c95327b160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:22.205787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969cd44-d93d-4321-9b4e-46c95327b160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:22.207194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969cd44-d93d-4321-9b4e-46c95327b160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:22.207635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5969cd44-d93d-4321-9b4e-46c95327b160 admin admin] PATH_INFO: `/v3/projects/6e308f70145b4bac88524fabdacc47bd/users/b18da7d08a3945b48029141064c17180/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:22.237501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e9749655c0b48cb9841f7f1d4abd05e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:22.239650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e9749655c0b48cb9841f7f1d4abd05e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:22.271660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969cd44-d93d-4321-9b4e-46c95327b160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e308f70145b4bac88524fabdacc47bd, user_id=b18da7d08a3945b48029141064c17180, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:22.273648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e9749655c0b48cb9841f7f1d4abd05e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:22.276815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5969cd44-d93d-4321-9b4e-46c95327b160 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:22.320894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e9749655c0b48cb9841f7f1d4abd05e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:22.328947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-be5e3fdd-43a3-4267-96bf-255de242c94c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e9749655c0b48cb9841f7f1d4abd05e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:22.336506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 855/1725] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:22 2022] PUT /identity/v3/projects/6e308f70145b4bac88524fabdacc47bd/users/b18da7d08a3945b48029141064c17180/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:22.349277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75e878-df40-457e-a992-9673feda160a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:22.349557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75e878-df40-457e-a992-9673feda160a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:22.349833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef75e878-df40-457e-a992-9673feda160a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:22.399945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 871/1726] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:21 2022] DELETE /identity/v3/users/3e9749655c0b48cb9841f7f1d4abd05e => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:22.413980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c043bd-ab8b-464a-b3de-b57a22ffa0f8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:22.414293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c043bd-ab8b-464a-b3de-b57a22ffa0f8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:22.414570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c043bd-ab8b-464a-b3de-b57a22ffa0f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:22.442546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ef75e878-df40-457e-a992-9673feda160a None None] MFA Rules not processed for user `b18da7d08a3945b48029141064c17180`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:22.445477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ef75e878-df40-457e-a992-9673feda160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:22.453572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a3c043bd-ab8b-464a-b3de-b57a22ffa0f8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:22.459997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a3c043bd-ab8b-464a-b3de-b57a22ffa0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:22.679056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:22.813175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 872/1727] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 400 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:22.846253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c043bd-ab8b-464a-b3de-b57a22ffa0f8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:22.846756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 856/1728] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:22 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 498 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:22.848881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a3c043bd-ab8b-464a-b3de-b57a22ffa0f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:22.925557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c043bd-ab8b-464a-b3de-b57a22ffa0f8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:22.964184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5e06e-5975-4d44-8290-b7a7d1e52c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:22.966787 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5e06e-5975-4d44-8290-b7a7d1e52c84 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:22.967095 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5e06e-5975-4d44-8290-b7a7d1e52c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:22.967557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f5e06e-5975-4d44-8290-b7a7d1e52c84 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:22.968046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5e06e-5975-4d44-8290-b7a7d1e52c84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:22.981460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f5e06e-5975-4d44-8290-b7a7d1e52c84 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:23.070423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:23.108121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 873/1729] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:22 2022] POST /identity/v3/projects => generated 381 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:23.112803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75e878-df40-457e-a992-9673feda160a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:23.121350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5e06e-5975-4d44-8290-b7a7d1e52c84 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:23.124833 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-47f5e06e-5975-4d44-8290-b7a7d1e52c84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:23.149756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef75e878-df40-457e-a992-9673feda160a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:23.404269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f6eac-ece3-4be5-b434-6e424026d628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:23.405006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f6eac-ece3-4be5-b434-6e424026d628 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:23.405211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f6eac-ece3-4be5-b434-6e424026d628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:23.405636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24f6eac-ece3-4be5-b434-6e424026d628 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:23.413441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f6eac-ece3-4be5-b434-6e424026d628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:23.416717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24f6eac-ece3-4be5-b434-6e424026d628 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:23.417688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f5e06e-5975-4d44-8290-b7a7d1e52c84 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:23.430494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f74cc3-d5a4-4c43-90da-bd0b8773a7a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:23.431914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f74cc3-d5a4-4c43-90da-bd0b8773a7a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:23.432206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f74cc3-d5a4-4c43-90da-bd0b8773a7a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:23.434274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f74cc3-d5a4-4c43-90da-bd0b8773a7a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:23.469331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f74cc3-d5a4-4c43-90da-bd0b8773a7a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:23.471728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f74cc3-d5a4-4c43-90da-bd0b8773a7a4 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:23.493857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 857/1730] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:23 2022] POST /identity/v3/users => generated 297 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:23.508560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 874/1731] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:23.530512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f6eac-ece3-4be5-b434-6e424026d628 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:23.567346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24f6eac-ece3-4be5-b434-6e424026d628 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:23.584619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2aa3c-518e-455b-b385-f3fd7f50cb3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:23.585498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2aa3c-518e-455b-b385-f3fd7f50cb3c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:23.586047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2aa3c-518e-455b-b385-f3fd7f50cb3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:23.586777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2aa3c-518e-455b-b385-f3fd7f50cb3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:23.587527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2aa3c-518e-455b-b385-f3fd7f50cb3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:23.589391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2aa3c-518e-455b-b385-f3fd7f50cb3c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:23.620093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 858/1732] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:23.623406 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f74cc3-d5a4-4c43-90da-bd0b8773a7a4 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:23.637120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f74cc3-d5a4-4c43-90da-bd0b8773a7a4 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:23.689616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e92ae-971c-4e67-93cf-634e4d02f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:23.690376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e92ae-971c-4e67-93cf-634e4d02f7be admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:23.690956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e92ae-971c-4e67-93cf-634e4d02f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:23.691382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e92ae-971c-4e67-93cf-634e4d02f7be admin admin] PATH_INFO: `/v3/projects/fe1b3a69ea2b42788ac936f726149b32/users/c3670045a3a342e4b5320457c878a539/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:23.745145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e92ae-971c-4e67-93cf-634e4d02f7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1b3a69ea2b42788ac936f726149b32, user_id=c3670045a3a342e4b5320457c878a539, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:23.749537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e45e92ae-971c-4e67-93cf-634e4d02f7be admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:23.823627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 875/1733] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:23 2022] PUT /identity/v3/projects/fe1b3a69ea2b42788ac936f726149b32/users/c3670045a3a342e4b5320457c878a539/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:23.827961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2aa3c-518e-455b-b385-f3fd7f50cb3c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:23.844099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2aa3c-518e-455b-b385-f3fd7f50cb3c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:24.118900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a83a91-046c-41e3-a433-96b53667ecba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:24.119994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a83a91-046c-41e3-a433-96b53667ecba admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:24.120419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a83a91-046c-41e3-a433-96b53667ecba admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:24.120837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a83a91-046c-41e3-a433-96b53667ecba admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:24.121644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a83a91-046c-41e3-a433-96b53667ecba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:24.123534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a83a91-046c-41e3-a433-96b53667ecba admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:24.147390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 859/1734] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:23 2022] GET /identity/v3/roles => generated 1420 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:24.152033 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e92ae-971c-4e67-93cf-634e4d02f7be admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:24.186317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e92ae-971c-4e67-93cf-634e4d02f7be admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:24.234978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fab2d6-c29e-4445-88b2-c765215ccd08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:24.235791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fab2d6-c29e-4445-88b2-c765215ccd08 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:24.236137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fab2d6-c29e-4445-88b2-c765215ccd08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:24.236817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fab2d6-c29e-4445-88b2-c765215ccd08 admin admin] PATH_INFO: `/v3/projects/fe1b3a69ea2b42788ac936f726149b32/users/c3670045a3a342e4b5320457c878a539/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:24.243998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fab2d6-c29e-4445-88b2-c765215ccd08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1b3a69ea2b42788ac936f726149b32, user_id=c3670045a3a342e4b5320457c878a539, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:24.247046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fab2d6-c29e-4445-88b2-c765215ccd08 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:24.276505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 876/1735] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:24 2022] PUT /identity/v3/projects/fe1b3a69ea2b42788ac936f726149b32/users/c3670045a3a342e4b5320457c878a539/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:24.288726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3f527-31fc-4f57-b0fb-792824cb31eb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:24.289386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3f527-31fc-4f57-b0fb-792824cb31eb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:24.289386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d3f527-31fc-4f57-b0fb-792824cb31eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:24.420091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25444286-5640-491a-a180-a35fcfdaf0d7 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:24.420874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25444286-5640-491a-a180-a35fcfdaf0d7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:24.421290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25444286-5640-491a-a180-a35fcfdaf0d7 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:24.423218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 877/1736] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:01:24.427095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-22d3f527-31fc-4f57-b0fb-792824cb31eb None None] MFA Rules not processed for user `c3670045a3a342e4b5320457c878a539`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:24.444777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-22d3f527-31fc-4f57-b0fb-792824cb31eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:24.574656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61795a9f-1a24-49d6-b0f9-318ac7bcb230 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:24.575183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61795a9f-1a24-49d6-b0f9-318ac7bcb230 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:24.575622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61795a9f-1a24-49d6-b0f9-318ac7bcb230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:24.651293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-61795a9f-1a24-49d6-b0f9-318ac7bcb230 None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:24.661179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-61795a9f-1a24-49d6-b0f9-318ac7bcb230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:24.665595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:24.781201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 860/1737] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:24 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 493 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:24.795547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3f527-31fc-4f57-b0fb-792824cb31eb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:24.807042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-22d3f527-31fc-4f57-b0fb-792824cb31eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:24.992690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 878/1738] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:01:24 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:24.993923 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d3f527-31fc-4f57-b0fb-792824cb31eb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:25.017471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abc68f-8e3a-4c89-b7d7-0f00b26b36b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:25.018747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abc68f-8e3a-4c89-b7d7-0f00b26b36b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:25.019121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abc68f-8e3a-4c89-b7d7-0f00b26b36b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:25.019490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abc68f-8e3a-4c89-b7d7-0f00b26b36b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:25.056815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abc68f-8e3a-4c89-b7d7-0f00b26b36b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:25.062349 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abc68f-8e3a-4c89-b7d7-0f00b26b36b0 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:25.125430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 861/1739] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:24 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:25.402642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61795a9f-1a24-49d6-b0f9-318ac7bcb230 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:25.415744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61795a9f-1a24-49d6-b0f9-318ac7bcb230 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:25.421153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b13289-3771-4349-aff0-00512b3de202 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:25.421993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b13289-3771-4349-aff0-00512b3de202 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:25.422512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b13289-3771-4349-aff0-00512b3de202 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:25.423472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b13289-3771-4349-aff0-00512b3de202 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:25.526011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:25.543387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b13289-3771-4349-aff0-00512b3de202 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:25.544972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b13289-3771-4349-aff0-00512b3de202 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:25.556993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 879/1740] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:25 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:26.123463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abc68f-8e3a-4c89-b7d7-0f00b26b36b0 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:26.125700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-29abc68f-8e3a-4c89-b7d7-0f00b26b36b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:26.151630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abc68f-8e3a-4c89-b7d7-0f00b26b36b0 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:26.159882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a85be7-547a-4ff2-ad69-814fb43e3141 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:26.160761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a85be7-547a-4ff2-ad69-814fb43e3141 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:26.161186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a85be7-547a-4ff2-ad69-814fb43e3141 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:26.161621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a85be7-547a-4ff2-ad69-814fb43e3141 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:26.174927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a85be7-547a-4ff2-ad69-814fb43e3141 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:26.178411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a85be7-547a-4ff2-ad69-814fb43e3141 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:26.194654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 862/1741] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:26.260910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b13289-3771-4349-aff0-00512b3de202 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:26.263724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-81b13289-3771-4349-aff0-00512b3de202 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:26.565867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b13289-3771-4349-aff0-00512b3de202 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:26.576389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:26.577629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:26.578191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:26.578455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] PATH_INFO: `/v3/projects/fea098e26a964973a3f9db0e35ad3e24` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:26.580935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fea098e26a964973a3f9db0e35ad3e24)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:26.584071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:26.668902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fea098e26a964973a3f9db0e35ad3e24', 'request_id': 'req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:26.680295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fea098e26a964973a3f9db0e35ad3e24', 'request_id': 'req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:26.688498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fea098e26a964973a3f9db0e35ad3e24', 'request_id': 'req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:26.705540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] The token cache is being invalidate because project fea098e26a964973a3f9db0e35ad3e24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:26.706172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:26.707168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c1e2ffcb-bf05-4c62-b42e-d9220f7b109f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:26.709348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 880/1742] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:26 2022] DELETE /identity/v3/projects/fea098e26a964973a3f9db0e35ad3e24 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:26.713280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a85be7-547a-4ff2-ad69-814fb43e3141 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:26.725779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a85be7-547a-4ff2-ad69-814fb43e3141 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:26.760046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7497ee9-e32d-4e6a-a184-ee7bb8d91008 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:26.760939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7497ee9-e32d-4e6a-a184-ee7bb8d91008 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:26.761531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7497ee9-e32d-4e6a-a184-ee7bb8d91008 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:26.763151 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 881/1743] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:01:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 13 07:01:26.794788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe1521-b72c-4e5f-a983-3e60ef506028 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:26.795131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe1521-b72c-4e5f-a983-3e60ef506028 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:26.795556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fe1521-b72c-4e5f-a983-3e60ef506028 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:26.821303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-71fe1521-b72c-4e5f-a983-3e60ef506028 None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:26.822845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-71fe1521-b72c-4e5f-a983-3e60ef506028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:26.870395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:26.871354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:26.871842 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:26.872343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] PATH_INFO: `/v3/users/2ab9f38bc5ad464484b195bc4f20dece` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:26.873977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ab9f38bc5ad464484b195bc4f20dece)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:26.876187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:26.944070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 882/1744] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:01:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:26.977989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ab9f38bc5ad464484b195bc4f20dece'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:26.984517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe1521-b72c-4e5f-a983-3e60ef506028 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:26.986159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-71fe1521-b72c-4e5f-a983-3e60ef506028 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:26.992022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ab9f38bc5ad464484b195bc4f20dece'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:27.024143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fe1521-b72c-4e5f-a983-3e60ef506028 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:27.027962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:27.035094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ab9f38bc5ad464484b195bc4f20dece'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:27.037420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ab9f38bc5ad464484b195bc4f20dece'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:27.041747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b73ad-4323-4742-894d-76fc857385d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:27.042459 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b73ad-4323-4742-894d-76fc857385d5 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:27.042741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b73ad-4323-4742-894d-76fc857385d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:27.043020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6b73ad-4323-4742-894d-76fc857385d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:27.056195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b73ad-4323-4742-894d-76fc857385d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:27.057796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6b73ad-4323-4742-894d-76fc857385d5 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:27.065478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ab9f38bc5ad464484b195bc4f20dece'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:27.066727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:27.073767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 863/1745] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:26 2022] DELETE /identity/v3/users/2ab9f38bc5ad464484b195bc4f20dece => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:27.076209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 883/1746] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:27.829556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:27.831846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:28.095268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b7506-8b0f-4224-92b1-ed1a62939fc4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:28.101382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81944646-f2ff-480a-949e-9851015e2a2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:28.103347 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81944646-f2ff-480a-949e-9851015e2a2a service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:28.103867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81944646-f2ff-480a-949e-9851015e2a2a service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:28.104822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81944646-f2ff-480a-949e-9851015e2a2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:28.113825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-81944646-f2ff-480a-949e-9851015e2a2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:28.421390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81944646-f2ff-480a-949e-9851015e2a2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:28.429392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81944646-f2ff-480a-949e-9851015e2a2a service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:28.465647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 864/1747] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:27 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:28.723654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b73ad-4323-4742-894d-76fc857385d5 service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:28.730183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1e6b73ad-4323-4742-894d-76fc857385d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:28.770659 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6b73ad-4323-4742-894d-76fc857385d5 service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:29.024982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:29.024982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:29.024982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:29.024982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] PATH_INFO: `/v3/projects/474b37a67c7746eaad6c7e01d1474a2c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:29.026226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=474b37a67c7746eaad6c7e01d1474a2c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:29.028226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:29.203893 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '474b37a67c7746eaad6c7e01d1474a2c', 'request_id': 'req-2d0a3d9e-a995-446e-9db1-d4f95012bab2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:29.227929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '474b37a67c7746eaad6c7e01d1474a2c', 'request_id': 'req-2d0a3d9e-a995-446e-9db1-d4f95012bab2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:29.247464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '474b37a67c7746eaad6c7e01d1474a2c', 'request_id': 'req-2d0a3d9e-a995-446e-9db1-d4f95012bab2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:29.312339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] The token cache is being invalidate because project 474b37a67c7746eaad6c7e01d1474a2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:29.315042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:29.318399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2d0a3d9e-a995-446e-9db1-d4f95012bab2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:29.328072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 884/1748] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:28 2022] DELETE /identity/v3/projects/474b37a67c7746eaad6c7e01d1474a2c => generated 0 bytes in 605 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:29.334550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a61e-59af-4b9b-a0a6-0acaa670ba72 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:29.335206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a61e-59af-4b9b-a0a6-0acaa670ba72 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:29.335206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7156a61e-59af-4b9b-a0a6-0acaa670ba72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:29.452642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7156a61e-59af-4b9b-a0a6-0acaa670ba72 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:29.454709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7156a61e-59af-4b9b-a0a6-0acaa670ba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:29.681731 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 865/1749] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:29.707955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86945b-4cec-4e90-9c84-327d7b9aa3b7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:29.707955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86945b-4cec-4e90-9c84-327d7b9aa3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:29.707955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa86945b-4cec-4e90-9c84-327d7b9aa3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:29.731836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-aa86945b-4cec-4e90-9c84-327d7b9aa3b7 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:29.734468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aa86945b-4cec-4e90-9c84-327d7b9aa3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:29.755796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 885/1750] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:29.760302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a61e-59af-4b9b-a0a6-0acaa670ba72 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:29.781572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7156a61e-59af-4b9b-a0a6-0acaa670ba72 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:29.794630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90957c6c-adcc-4490-9e79-98da8e38dede admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:29.795560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90957c6c-adcc-4490-9e79-98da8e38dede admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:29.797465 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90957c6c-adcc-4490-9e79-98da8e38dede admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:29.800243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90957c6c-adcc-4490-9e79-98da8e38dede admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:29.801394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90957c6c-adcc-4490-9e79-98da8e38dede admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:29.802855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90957c6c-adcc-4490-9e79-98da8e38dede admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:29.807521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90957c6c-adcc-4490-9e79-98da8e38dede admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:29.840354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 866/1751] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:29.845961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86945b-4cec-4e90-9c84-327d7b9aa3b7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:29.867947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa86945b-4cec-4e90-9c84-327d7b9aa3b7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:29.880807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831c57b-4172-479b-9617-9bd05429f468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:29.882997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831c57b-4172-479b-9617-9bd05429f468 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:29.883504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831c57b-4172-479b-9617-9bd05429f468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:29.884016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3831c57b-4172-479b-9617-9bd05429f468 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:29.885016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831c57b-4172-479b-9617-9bd05429f468 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:29.887643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3831c57b-4172-479b-9617-9bd05429f468 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:29.940898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90957c6c-adcc-4490-9e79-98da8e38dede admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:29.960084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 886/1752] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:29 2022] POST /identity/v3/projects => generated 365 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:01:29.963787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90957c6c-adcc-4490-9e79-98da8e38dede admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:30.004131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:30.110648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89d698-6d81-435c-b592-b4891e6dcb89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:30.114608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89d698-6d81-435c-b592-b4891e6dcb89 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:30.115007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89d698-6d81-435c-b592-b4891e6dcb89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:30.115290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e89d698-6d81-435c-b592-b4891e6dcb89 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:30.117027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89d698-6d81-435c-b592-b4891e6dcb89 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:30.122380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e89d698-6d81-435c-b592-b4891e6dcb89 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:30.187058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831c57b-4172-479b-9617-9bd05429f468 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:30.200845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 867/1753] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:29 2022] POST /identity/v3/users => generated 283 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:30.207567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3831c57b-4172-479b-9617-9bd05429f468 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:30.221869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23a8f6-dc21-45a3-8a3f-6b338f989325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:30.222420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23a8f6-dc21-45a3-8a3f-6b338f989325 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:30.222706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23a8f6-dc21-45a3-8a3f-6b338f989325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:30.222985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a23a8f6-dc21-45a3-8a3f-6b338f989325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:30.225809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23a8f6-dc21-45a3-8a3f-6b338f989325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:30.230590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a23a8f6-dc21-45a3-8a3f-6b338f989325 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:30.263371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89d698-6d81-435c-b592-b4891e6dcb89 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:30.273938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 887/1754] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:30.283957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e89d698-6d81-435c-b592-b4891e6dcb89 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:30.296976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c456f43-73af-48cb-9903-2ec38e20526d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:30.298566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c456f43-73af-48cb-9903-2ec38e20526d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:30.298870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c456f43-73af-48cb-9903-2ec38e20526d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:30.301910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c456f43-73af-48cb-9903-2ec38e20526d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:30.308005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c456f43-73af-48cb-9903-2ec38e20526d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:30.314269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c456f43-73af-48cb-9903-2ec38e20526d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:30.349566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 868/1755] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:30 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:30.351729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23a8f6-dc21-45a3-8a3f-6b338f989325 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:30.375661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a23a8f6-dc21-45a3-8a3f-6b338f989325 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:30.444336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf953231-3428-4676-8bcc-779841b67436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:30.445075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf953231-3428-4676-8bcc-779841b67436 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:30.445430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf953231-3428-4676-8bcc-779841b67436 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:30.445778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf953231-3428-4676-8bcc-779841b67436 admin admin] PATH_INFO: `/v3/projects/0e826c1b0d5b450babb5adfbb4247ca9/users/65223f6782964aad97670fa022581ef7/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:30.465910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf953231-3428-4676-8bcc-779841b67436 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e826c1b0d5b450babb5adfbb4247ca9, user_id=65223f6782964aad97670fa022581ef7, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:30.468776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf953231-3428-4676-8bcc-779841b67436 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:30.502520 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 888/1756] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:30 2022] PUT /identity/v3/projects/0e826c1b0d5b450babb5adfbb4247ca9/users/65223f6782964aad97670fa022581ef7/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:30.512223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efd3db-a3c9-4a24-ba19-37a07e0f4c27 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:30.512868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efd3db-a3c9-4a24-ba19-37a07e0f4c27 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:30.513149 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84efd3db-a3c9-4a24-ba19-37a07e0f4c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:30.625833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-84efd3db-a3c9-4a24-ba19-37a07e0f4c27 None None] MFA Rules not processed for user `65223f6782964aad97670fa022581ef7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:30.631006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-84efd3db-a3c9-4a24-ba19-37a07e0f4c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:30.774127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 869/1757] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:30.778296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf953231-3428-4676-8bcc-779841b67436 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:30.817663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:30.868541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf953231-3428-4676-8bcc-779841b67436 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:30.879531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ffb65-dcf9-4e1a-825e-9d9e606fc8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:30.880725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ffb65-dcf9-4e1a-825e-9d9e606fc8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:30.881013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ffb65-dcf9-4e1a-825e-9d9e606fc8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:30.881589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ffb65-dcf9-4e1a-825e-9d9e606fc8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:30.882377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ffb65-dcf9-4e1a-825e-9d9e606fc8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:30.884345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2ffb65-dcf9-4e1a-825e-9d9e606fc8b6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:30.940972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 889/1758] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:30 2022] POST /identity/v3/projects => generated 365 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:30.941866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efd3db-a3c9-4a24-ba19-37a07e0f4c27 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:30.961357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84efd3db-a3c9-4a24-ba19-37a07e0f4c27 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:31.090920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c3353-82db-42f1-8d43-36942df0eeb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:31.091564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c3353-82db-42f1-8d43-36942df0eeb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:31.092454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c3353-82db-42f1-8d43-36942df0eeb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:31.092699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56c3353-82db-42f1-8d43-36942df0eeb9 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:31.093530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c3353-82db-42f1-8d43-36942df0eeb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:31.098023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56c3353-82db-42f1-8d43-36942df0eeb9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:31.131585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 870/1759] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:30 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:01:31.136106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ffb65-dcf9-4e1a-825e-9d9e606fc8b6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:31.149578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ffb65-dcf9-4e1a-825e-9d9e606fc8b6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:31.154537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f2192-f9bb-4336-9d2e-5d8904af3d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:31.155259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f2192-f9bb-4336-9d2e-5d8904af3d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:31.155663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f2192-f9bb-4336-9d2e-5d8904af3d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:31.155954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-228f2192-f9bb-4336-9d2e-5d8904af3d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:31.156701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f2192-f9bb-4336-9d2e-5d8904af3d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:31.158768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-228f2192-f9bb-4336-9d2e-5d8904af3d3d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:31.178007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 890/1760] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:31.183124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c3353-82db-42f1-8d43-36942df0eeb9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:31.209619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56c3353-82db-42f1-8d43-36942df0eeb9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:31.214913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54fe83-b22b-47a5-a608-08c55a99de40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:31.215604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54fe83-b22b-47a5-a608-08c55a99de40 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:31.215888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54fe83-b22b-47a5-a608-08c55a99de40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:31.216150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e54fe83-b22b-47a5-a608-08c55a99de40 admin admin] PATH_INFO: `/v3/projects/800c3972fcc7423d86ec84f1c339a477/users/45727c95cbda42c5b0ab5aa144bb5689/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:31.235273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54fe83-b22b-47a5-a608-08c55a99de40 admin admin] RBAC: Authorizing `identity:create_grant(project_id=800c3972fcc7423d86ec84f1c339a477, user_id=45727c95cbda42c5b0ab5aa144bb5689, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:31.238490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e54fe83-b22b-47a5-a608-08c55a99de40 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:31.253991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:31.263128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 871/1761] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:31 2022] PUT /identity/v3/projects/800c3972fcc7423d86ec84f1c339a477/users/45727c95cbda42c5b0ab5aa144bb5689/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:31.264393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f2192-f9bb-4336-9d2e-5d8904af3d3d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:31.277059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-228f2192-f9bb-4336-9d2e-5d8904af3d3d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:31.366623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c41a60-0c04-4421-aef7-c66981c2a206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:31.367282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c41a60-0c04-4421-aef7-c66981c2a206 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:31.367564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c41a60-0c04-4421-aef7-c66981c2a206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:31.367834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c41a60-0c04-4421-aef7-c66981c2a206 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:31.368490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c41a60-0c04-4421-aef7-c66981c2a206 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:31.370388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c41a60-0c04-4421-aef7-c66981c2a206 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:31.383050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 891/1762] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:31 2022] GET /identity/v3/roles => generated 1420 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:31.386740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54fe83-b22b-47a5-a608-08c55a99de40 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:31.409211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e54fe83-b22b-47a5-a608-08c55a99de40 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:31.415261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1652714-0be3-4dcc-8959-e9969ab37ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:31.415958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1652714-0be3-4dcc-8959-e9969ab37ab0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:31.416240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1652714-0be3-4dcc-8959-e9969ab37ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:31.416519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1652714-0be3-4dcc-8959-e9969ab37ab0 admin admin] PATH_INFO: `/v3/projects/800c3972fcc7423d86ec84f1c339a477/users/45727c95cbda42c5b0ab5aa144bb5689/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:31.418491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1652714-0be3-4dcc-8959-e9969ab37ab0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=800c3972fcc7423d86ec84f1c339a477, user_id=45727c95cbda42c5b0ab5aa144bb5689, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:31.421256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1652714-0be3-4dcc-8959-e9969ab37ab0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:31.433632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 872/1763] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:31 2022] PUT /identity/v3/projects/800c3972fcc7423d86ec84f1c339a477/users/45727c95cbda42c5b0ab5aa144bb5689/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:31.444824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b2a22-38f6-4f9a-a26a-48703706b3dc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:31.445108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b2a22-38f6-4f9a-a26a-48703706b3dc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:31.445385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49b2a22-38f6-4f9a-a26a-48703706b3dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:31.485284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f49b2a22-38f6-4f9a-a26a-48703706b3dc None None] MFA Rules not processed for user `45727c95cbda42c5b0ab5aa144bb5689`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:31.486791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f49b2a22-38f6-4f9a-a26a-48703706b3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:31.588846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 892/1764] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:31 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:31.601024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1652714-0be3-4dcc-8959-e9969ab37ab0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:31.613397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a1652714-0be3-4dcc-8959-e9969ab37ab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:31.707818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1652714-0be3-4dcc-8959-e9969ab37ab0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:31.714005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b402ed-c5e8-44e4-be92-e38793806347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:31.715055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b402ed-c5e8-44e4-be92-e38793806347 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:31.715352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b402ed-c5e8-44e4-be92-e38793806347 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:31.715703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b402ed-c5e8-44e4-be92-e38793806347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:31.727202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b402ed-c5e8-44e4-be92-e38793806347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:31.728646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b402ed-c5e8-44e4-be92-e38793806347 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:31.739966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 873/1765] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:31 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:31.823824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b2a22-38f6-4f9a-a26a-48703706b3dc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:31.825392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f49b2a22-38f6-4f9a-a26a-48703706b3dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:31.838380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49b2a22-38f6-4f9a-a26a-48703706b3dc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:31.842942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4549e7d-3412-4b8d-8c4d-60a466b76edd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:31.843694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4549e7d-3412-4b8d-8c4d-60a466b76edd service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:31.843972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4549e7d-3412-4b8d-8c4d-60a466b76edd service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:31.844247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4549e7d-3412-4b8d-8c4d-60a466b76edd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:31.864416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:31.909125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4549e7d-3412-4b8d-8c4d-60a466b76edd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:31.910542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4549e7d-3412-4b8d-8c4d-60a466b76edd service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:31.919888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 893/1766] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:36.951329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b402ed-c5e8-44e4-be92-e38793806347 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:37.046631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b402ed-c5e8-44e4-be92-e38793806347 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:37.053052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:37.053998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:37.054748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:37.055250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] PATH_INFO: `/v3/users/65223f6782964aad97670fa022581ef7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:37.057278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=65223f6782964aad97670fa022581ef7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:37.065509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:37.172940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65223f6782964aad97670fa022581ef7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:37.187281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65223f6782964aad97670fa022581ef7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:37.207893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65223f6782964aad97670fa022581ef7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:37.209362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65223f6782964aad97670fa022581ef7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:37.218298 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65223f6782964aad97670fa022581ef7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:37.231722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 874/1767] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:36 2022] DELETE /identity/v3/users/65223f6782964aad97670fa022581ef7 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:37.237209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5edf3-28eb-4a9f-8c77-138c52ca980f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:37.237747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5edf3-28eb-4a9f-8c77-138c52ca980f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:37.237822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e5edf3-28eb-4a9f-8c77-138c52ca980f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:37.262113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-42e5edf3-28eb-4a9f-8c77-138c52ca980f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:37.263982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-42e5edf3-28eb-4a9f-8c77-138c52ca980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:37.392722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 894/1768] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:37.410968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:37.413987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:37.442784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:37.577059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b20e0b-f8b9-471f-9f44-0604662a78fd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:37.587497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49ad24-ea31-4851-9821-d1b770dbc38f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:37.588262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49ad24-ea31-4851-9821-d1b770dbc38f service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:37.588548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49ad24-ea31-4851-9821-d1b770dbc38f service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:37.588800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f49ad24-ea31-4851-9821-d1b770dbc38f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:37.619097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49ad24-ea31-4851-9821-d1b770dbc38f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:37.620746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f49ad24-ea31-4851-9821-d1b770dbc38f service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:37.636795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 875/1769] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:37.982447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5edf3-28eb-4a9f-8c77-138c52ca980f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:37.983947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-42e5edf3-28eb-4a9f-8c77-138c52ca980f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:37.997031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e5edf3-28eb-4a9f-8c77-138c52ca980f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:38.002996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:38.004254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:38.004684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:38.005465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] PATH_INFO: `/v3/projects/0e826c1b0d5b450babb5adfbb4247ca9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:38.007758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e826c1b0d5b450babb5adfbb4247ca9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:38.010455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:38.100165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e826c1b0d5b450babb5adfbb4247ca9', 'request_id': 'req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.107995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e826c1b0d5b450babb5adfbb4247ca9', 'request_id': 'req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.114932 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e826c1b0d5b450babb5adfbb4247ca9', 'request_id': 'req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.135509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] The token cache is being invalidate because project 0e826c1b0d5b450babb5adfbb4247ca9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:38.136049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.137041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.139276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 895/1770] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:37 2022] DELETE /identity/v3/projects/0e826c1b0d5b450babb5adfbb4247ca9 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:38.143308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49ad24-ea31-4851-9821-d1b770dbc38f service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:38.155536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f49ad24-ea31-4851-9821-d1b770dbc38f service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:38.233286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:38.233963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:38.234277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:38.234552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] PATH_INFO: `/v3/users/45727c95cbda42c5b0ab5aa144bb5689` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:38.235749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] RBAC: Authorizing `identity:delete_user(user_id=45727c95cbda42c5b0ab5aa144bb5689)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:38.237818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:38.295264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45727c95cbda42c5b0ab5aa144bb5689'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.309851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45727c95cbda42c5b0ab5aa144bb5689'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.327296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45727c95cbda42c5b0ab5aa144bb5689'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.328088 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45727c95cbda42c5b0ab5aa144bb5689'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.336163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7159baff-2b94-4fca-8f12-d4f13891153f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45727c95cbda42c5b0ab5aa144bb5689'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.342518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 876/1771] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:38 2022] DELETE /identity/v3/users/45727c95cbda42c5b0ab5aa144bb5689 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:38.653489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:38.655035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:38.668182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f6e6c-1a14-44af-9ca7-90126bfbc76e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:38.733094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:38.752742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:38.753398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:38.753998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:38.753998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] PATH_INFO: `/v3/projects/800c3972fcc7423d86ec84f1c339a477` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:38.755216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=800c3972fcc7423d86ec84f1c339a477)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:38.757405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:38.831376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '800c3972fcc7423d86ec84f1c339a477', 'request_id': 'req-741019e3-6415-46ba-93b7-af77caf536ec'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.839561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '800c3972fcc7423d86ec84f1c339a477', 'request_id': 'req-741019e3-6415-46ba-93b7-af77caf536ec'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.847435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '800c3972fcc7423d86ec84f1c339a477', 'request_id': 'req-741019e3-6415-46ba-93b7-af77caf536ec'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.864464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] The token cache is being invalidate because project 800c3972fcc7423d86ec84f1c339a477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:38.864866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.865597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-741019e3-6415-46ba-93b7-af77caf536ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:38.871430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 896/1772] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:38 2022] DELETE /identity/v3/projects/800c3972fcc7423d86ec84f1c339a477 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:38.873003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1770c-4943-4f70-ad36-72aeaaf70181 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:38.873546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1770c-4943-4f70-ad36-72aeaaf70181 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:38.873993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1770c-4943-4f70-ad36-72aeaaf70181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:38.899391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2ce1770c-4943-4f70-ad36-72aeaaf70181 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:38.900783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2ce1770c-4943-4f70-ad36-72aeaaf70181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:38.932801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:39.000333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73627b37-8b5d-4802-9692-56a7d49e1fa1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:39.000892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73627b37-8b5d-4802-9692-56a7d49e1fa1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:39.001270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73627b37-8b5d-4802-9692-56a7d49e1fa1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:39.005964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 877/1773] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:39.024708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-73627b37-8b5d-4802-9692-56a7d49e1fa1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:39.026105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-73627b37-8b5d-4802-9692-56a7d49e1fa1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:39.035543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 897/1774] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:39.040825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1770c-4943-4f70-ad36-72aeaaf70181 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:39.054015 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1770c-4943-4f70-ad36-72aeaaf70181 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:39.064282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32d85c-7905-4ad5-b457-9501a9f210f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:39.065357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32d85c-7905-4ad5-b457-9501a9f210f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:39.065503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32d85c-7905-4ad5-b457-9501a9f210f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:39.065881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae32d85c-7905-4ad5-b457-9501a9f210f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:39.066720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32d85c-7905-4ad5-b457-9501a9f210f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:39.067151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32d85c-7905-4ad5-b457-9501a9f210f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:39.070134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae32d85c-7905-4ad5-b457-9501a9f210f4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:39.089308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73627b37-8b5d-4802-9692-56a7d49e1fa1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:39.095868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 878/1775] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:39.100924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73627b37-8b5d-4802-9692-56a7d49e1fa1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:39.108132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aad90-d2a7-4ab4-a78a-6a466a901cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:39.109076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aad90-d2a7-4ab4-a78a-6a466a901cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:39.109735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aad90-d2a7-4ab4-a78a-6a466a901cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:39.110292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aad90-d2a7-4ab4-a78a-6a466a901cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:39.112386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aad90-d2a7-4ab4-a78a-6a466a901cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:39.115759 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aad90-d2a7-4ab4-a78a-6a466a901cb9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:39.184078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 898/1776] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:39 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:39.184762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32d85c-7905-4ad5-b457-9501a9f210f4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:39.215321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae32d85c-7905-4ad5-b457-9501a9f210f4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:39.303626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf8faa-e7b0-45ef-a093-ab66185d3e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:39.304315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8faa-e7b0-45ef-a093-ab66185d3e94 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:39.304824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8faa-e7b0-45ef-a093-ab66185d3e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:39.305312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bf8faa-e7b0-45ef-a093-ab66185d3e94 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:39.306581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf8faa-e7b0-45ef-a093-ab66185d3e94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:39.309365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8bf8faa-e7b0-45ef-a093-ab66185d3e94 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:39.341056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 879/1777] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:39 2022] POST /identity/v3/users => generated 294 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:01:39.348023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aad90-d2a7-4ab4-a78a-6a466a901cb9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:39.368271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aad90-d2a7-4ab4-a78a-6a466a901cb9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:39.379445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cc650-b1a2-48ea-b278-9905447f8edb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:39.380677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cc650-b1a2-48ea-b278-9905447f8edb admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:39.383447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cc650-b1a2-48ea-b278-9905447f8edb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:39.383962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4cc650-b1a2-48ea-b278-9905447f8edb admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:39.388287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cc650-b1a2-48ea-b278-9905447f8edb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:39.390107 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4cc650-b1a2-48ea-b278-9905447f8edb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:39.410636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 899/1778] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:39.415480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf8faa-e7b0-45ef-a093-ab66185d3e94 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:39.430571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bf8faa-e7b0-45ef-a093-ab66185d3e94 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:39.467886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b34fc9-94d3-41bb-bf03-610af4b3664a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:39.468868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b34fc9-94d3-41bb-bf03-610af4b3664a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:39.469655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b34fc9-94d3-41bb-bf03-610af4b3664a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:39.470211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b34fc9-94d3-41bb-bf03-610af4b3664a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:39.472769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b34fc9-94d3-41bb-bf03-610af4b3664a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:39.473487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b34fc9-94d3-41bb-bf03-610af4b3664a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:39.496771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 880/1779] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:39 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:39.501459 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cc650-b1a2-48ea-b278-9905447f8edb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:39.516608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4cc650-b1a2-48ea-b278-9905447f8edb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:39.524187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397a196-8be3-48dd-94ad-ea332a04f421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:39.525284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397a196-8be3-48dd-94ad-ea332a04f421 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:39.525792 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397a196-8be3-48dd-94ad-ea332a04f421 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:39.526321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6397a196-8be3-48dd-94ad-ea332a04f421 admin admin] PATH_INFO: `/v3/projects/1cea8a7dac1342b8906bfe039916b617/users/6da40f665b304c0a8c3512f024d1481d/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:39.544291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397a196-8be3-48dd-94ad-ea332a04f421 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1cea8a7dac1342b8906bfe039916b617, user_id=6da40f665b304c0a8c3512f024d1481d, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:39.547185 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6397a196-8be3-48dd-94ad-ea332a04f421 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:39.564646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 900/1780] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:39 2022] PUT /identity/v3/projects/1cea8a7dac1342b8906bfe039916b617/users/6da40f665b304c0a8c3512f024d1481d/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:39.576776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f940f-4e8a-4881-80f7-d6a3f5273899 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:39.577230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f940f-4e8a-4881-80f7-d6a3f5273899 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:39.577569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f940f-4e8a-4881-80f7-d6a3f5273899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:39.653165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b90f940f-4e8a-4881-80f7-d6a3f5273899 None None] MFA Rules not processed for user `6da40f665b304c0a8c3512f024d1481d`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:39.654657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b90f940f-4e8a-4881-80f7-d6a3f5273899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:39.765707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 881/1781] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:39 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:39.770013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397a196-8be3-48dd-94ad-ea332a04f421 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:39.874287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6397a196-8be3-48dd-94ad-ea332a04f421 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:39.880866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483b03-fd61-4164-8599-409ecf993e05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:39.881544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483b03-fd61-4164-8599-409ecf993e05 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:39.881832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483b03-fd61-4164-8599-409ecf993e05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:39.882134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94483b03-fd61-4164-8599-409ecf993e05 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:39.882855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94483b03-fd61-4164-8599-409ecf993e05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:39.884880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94483b03-fd61-4164-8599-409ecf993e05 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:39.903175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:39.926124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 901/1782] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:39 2022] POST /identity/v3/projects => generated 387 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:39.926523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f940f-4e8a-4881-80f7-d6a3f5273899 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:39.940668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f940f-4e8a-4881-80f7-d6a3f5273899 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:39.975367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:40.055385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91262652-9de7-4fa5-9dde-58d5e5a597b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.056808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91262652-9de7-4fa5-9dde-58d5e5a597b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.057189 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91262652-9de7-4fa5-9dde-58d5e5a597b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.057540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91262652-9de7-4fa5-9dde-58d5e5a597b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.058987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91262652-9de7-4fa5-9dde-58d5e5a597b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.061466 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91262652-9de7-4fa5-9dde-58d5e5a597b2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.117120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 882/1783] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:39 2022] POST /identity/v3/users => generated 294 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:01:40.118984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483b03-fd61-4164-8599-409ecf993e05 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.131857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94483b03-fd61-4164-8599-409ecf993e05 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:40.137254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cba325-942e-4d25-beda-479a12a5d475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.138363 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cba325-942e-4d25-beda-479a12a5d475 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.138734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cba325-942e-4d25-beda-479a12a5d475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.139077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cba325-942e-4d25-beda-479a12a5d475 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.139831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cba325-942e-4d25-beda-479a12a5d475 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.142588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cba325-942e-4d25-beda-479a12a5d475 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.177489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 902/1784] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:40.182186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91262652-9de7-4fa5-9dde-58d5e5a597b2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.192019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91262652-9de7-4fa5-9dde-58d5e5a597b2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:40.196750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d91640b-bba0-4f8d-a6c4-3a77a08f00cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.197379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91640b-bba0-4f8d-a6c4-3a77a08f00cf admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.197660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91640b-bba0-4f8d-a6c4-3a77a08f00cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.197935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d91640b-bba0-4f8d-a6c4-3a77a08f00cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.198658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91640b-bba0-4f8d-a6c4-3a77a08f00cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.200592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d91640b-bba0-4f8d-a6c4-3a77a08f00cf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.214930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 883/1785] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:01:40.216554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cba325-942e-4d25-beda-479a12a5d475 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.230342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cba325-942e-4d25-beda-479a12a5d475 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:40.235613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06d440-e7fc-4084-a9f6-3f11789ec49a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.236326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06d440-e7fc-4084-a9f6-3f11789ec49a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.236604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06d440-e7fc-4084-a9f6-3f11789ec49a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.236847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee06d440-e7fc-4084-a9f6-3f11789ec49a admin admin] PATH_INFO: `/v3/projects/f9164cf694de45679c0e8d88badf9730/users/948d2845cb00403e83e1dc12db9ea8ec/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.253766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06d440-e7fc-4084-a9f6-3f11789ec49a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9164cf694de45679c0e8d88badf9730, user_id=948d2845cb00403e83e1dc12db9ea8ec, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.256609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee06d440-e7fc-4084-a9f6-3f11789ec49a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.271036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 903/1786] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:40 2022] PUT /identity/v3/projects/f9164cf694de45679c0e8d88badf9730/users/948d2845cb00403e83e1dc12db9ea8ec/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:40.282874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6d5fb-51a8-4ef6-abee-b867644a832b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.283164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6d5fb-51a8-4ef6-abee-b867644a832b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.283441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c6d5fb-51a8-4ef6-abee-b867644a832b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.313144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-44c6d5fb-51a8-4ef6-abee-b867644a832b None None] MFA Rules not processed for user `948d2845cb00403e83e1dc12db9ea8ec`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:40.314397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-44c6d5fb-51a8-4ef6-abee-b867644a832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:40.389748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 884/1787] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:40 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:40.394612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06d440-e7fc-4084-a9f6-3f11789ec49a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.485485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee06d440-e7fc-4084-a9f6-3f11789ec49a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:40.493090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81482cf4-8e47-4dae-afe7-d3f4ac4f78b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.494496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81482cf4-8e47-4dae-afe7-d3f4ac4f78b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.494932 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81482cf4-8e47-4dae-afe7-d3f4ac4f78b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.495338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81482cf4-8e47-4dae-afe7-d3f4ac4f78b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.496289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81482cf4-8e47-4dae-afe7-d3f4ac4f78b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.499029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81482cf4-8e47-4dae-afe7-d3f4ac4f78b0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.534405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6d5fb-51a8-4ef6-abee-b867644a832b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.536089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-44c6d5fb-51a8-4ef6-abee-b867644a832b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:40.546699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 904/1788] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:40 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:40.550691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81482cf4-8e47-4dae-afe7-d3f4ac4f78b0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.551002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c6d5fb-51a8-4ef6-abee-b867644a832b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:40.617247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.618451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.618778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.619060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] PATH_INFO: `/v3/users/565e45e3594244a1a1233455fa913f70` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.620873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=565e45e3594244a1a1233455fa913f70)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.623155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.643688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81482cf4-8e47-4dae-afe7-d3f4ac4f78b0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:40.650758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98375244-2b2e-4b1e-8fda-f92fa98b0904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.651445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98375244-2b2e-4b1e-8fda-f92fa98b0904 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.651729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98375244-2b2e-4b1e-8fda-f92fa98b0904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.652001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98375244-2b2e-4b1e-8fda-f92fa98b0904 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.652640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98375244-2b2e-4b1e-8fda-f92fa98b0904 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.656285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98375244-2b2e-4b1e-8fda-f92fa98b0904 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.678885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 905/1789] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:40 2022] POST /identity/v3/users => generated 300 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:40.679946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98375244-2b2e-4b1e-8fda-f92fa98b0904 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.681189 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:40.682023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-98375244-2b2e-4b1e-8fda-f92fa98b0904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:40.693895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '565e45e3594244a1a1233455fa913f70'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.695502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98375244-2b2e-4b1e-8fda-f92fa98b0904 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:40.702397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.703470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.703941 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.704289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '565e45e3594244a1a1233455fa913f70'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.704397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] PATH_INFO: `/v3/users/b18da7d08a3945b48029141064c17180` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.706212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b18da7d08a3945b48029141064c17180)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.709190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.720564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '565e45e3594244a1a1233455fa913f70'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.721395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '565e45e3594244a1a1233455fa913f70'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.730500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '565e45e3594244a1a1233455fa913f70'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.737382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 885/1790] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:40 2022] DELETE /identity/v3/users/565e45e3594244a1a1233455fa913f70 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:40.737936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.739311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:40.750896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:40.766449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b18da7d08a3945b48029141064c17180'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.767146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b18da7d08a3945b48029141064c17180'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.777277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b18da7d08a3945b48029141064c17180'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.792532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b18da7d08a3945b48029141064c17180'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.793797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0ce549b3-19c8-4595-8c32-85bdc50095f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b18da7d08a3945b48029141064c17180'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:40.809123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 906/1791] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:40 2022] DELETE /identity/v3/users/b18da7d08a3945b48029141064c17180 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:40.812372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcf8a9-2304-4b05-89a5-4e9522e5a3a4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.812669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcf8a9-2304-4b05-89a5-4e9522e5a3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.812951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedcf8a9-2304-4b05-89a5-4e9522e5a3a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.816080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5adf07-f479-404e-8499-1dad4656de77 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:40.840108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-bedcf8a9-2304-4b05-89a5-4e9522e5a3a4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:40.841972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bedcf8a9-2304-4b05-89a5-4e9522e5a3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:40.899780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b74af00-0b6e-4a08-bf12-fdc63489d8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.900687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74af00-0b6e-4a08-bf12-fdc63489d8e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.901034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74af00-0b6e-4a08-bf12-fdc63489d8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.901486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b74af00-0b6e-4a08-bf12-fdc63489d8e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.902299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74af00-0b6e-4a08-bf12-fdc63489d8e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.904127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b74af00-0b6e-4a08-bf12-fdc63489d8e3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.919225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 886/1792] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:40 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:40.920699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8dbd2-b449-49bc-bd32-7a5407870d00 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.921046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8dbd2-b449-49bc-bd32-7a5407870d00 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.921382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8dbd2-b449-49bc-bd32-7a5407870d00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.938125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 907/1793] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:40.938125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcf8a9-2304-4b05-89a5-4e9522e5a3a4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.938686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bedcf8a9-2304-4b05-89a5-4e9522e5a3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:40.949960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5cd8dbd2-b449-49bc-bd32-7a5407870d00 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:40.950968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedcf8a9-2304-4b05-89a5-4e9522e5a3a4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:40.953484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5cd8dbd2-b449-49bc-bd32-7a5407870d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:40.955983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:40.956631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:40.956909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:40.957287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] PATH_INFO: `/v3/users/aad48d2bb116464b92350939891956f0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:40.959105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] RBAC: Authorizing `identity:delete_user(user_id=aad48d2bb116464b92350939891956f0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:40.960905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:40.965486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 887/1794] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:40.966958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8dbd2-b449-49bc-bd32-7a5407870d00 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:40.968839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5cd8dbd2-b449-49bc-bd32-7a5407870d00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:40.996387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8dbd2-b449-49bc-bd32-7a5407870d00 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:41.002629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40c412-7268-4eae-8861-8a3826ef2591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:41.003531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40c412-7268-4eae-8861-8a3826ef2591 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:41.003814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40c412-7268-4eae-8861-8a3826ef2591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:41.004098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f40c412-7268-4eae-8861-8a3826ef2591 admin admin] PATH_INFO: `/v3/projects/beabf0655a17461d8f09e76132bdfc36/users/bc2cc0b39fdf476fa45fb845cf50cd8b/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:41.063658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40c412-7268-4eae-8861-8a3826ef2591 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beabf0655a17461d8f09e76132bdfc36, user_id=bc2cc0b39fdf476fa45fb845cf50cd8b, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:41.069619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f40c412-7268-4eae-8861-8a3826ef2591 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:41.073727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aad48d2bb116464b92350939891956f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:41.075125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aad48d2bb116464b92350939891956f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:41.095976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aad48d2bb116464b92350939891956f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:41.104199 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 888/1795] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:40 2022] PUT /identity/v3/projects/beabf0655a17461d8f09e76132bdfc36/users/bc2cc0b39fdf476fa45fb845cf50cd8b/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:41.105838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40c412-7268-4eae-8861-8a3826ef2591 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:41.111773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4f40c412-7268-4eae-8861-8a3826ef2591 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:41.122169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aad48d2bb116464b92350939891956f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:41.124085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aad48d2bb116464b92350939891956f0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:41.148413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 908/1796] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:40 2022] DELETE /identity/v3/users/aad48d2bb116464b92350939891956f0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:41.149588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:41.153436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:41.153681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:41.218456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f40c412-7268-4eae-8861-8a3826ef2591 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:41.227195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:41.299912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6795d5e2-632c-40ea-985b-62d53e4cd13f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:41.308379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e28ed1-7209-4336-b290-033723d3511c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:41.309737 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e28ed1-7209-4336-b290-033723d3511c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:41.310740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e28ed1-7209-4336-b290-033723d3511c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:41.311113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e28ed1-7209-4336-b290-033723d3511c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:41.313889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:41.322131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-47e28ed1-7209-4336-b290-033723d3511c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:41.325986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:41.327490 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:41.327880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:41.334694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:41.463253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:41.464949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:41.467337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e28ed1-7209-4336-b290-033723d3511c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:41.468665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e28ed1-7209-4336-b290-033723d3511c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:41.477613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 889/1797] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:41.478622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:41.480239 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:41.480848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 909/1798] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:41.482488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89aecf-2961-47bd-8c3f-bb2d9a5bf8c2 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:41.482888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89aecf-2961-47bd-8c3f-bb2d9a5bf8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:41.483219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b89aecf-2961-47bd-8c3f-bb2d9a5bf8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:41.494497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6eac4d0-b402-4a8c-a2b8-a69d03e8023a service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:41.501204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a5a57-0fd6-49b2-9d4b-ce23efda5f12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:41.501915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5a57-0fd6-49b2-9d4b-ce23efda5f12 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:41.502292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5a57-0fd6-49b2-9d4b-ce23efda5f12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:41.502630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a5a57-0fd6-49b2-9d4b-ce23efda5f12 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:41.503341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a5a57-0fd6-49b2-9d4b-ce23efda5f12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:41.505058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a5a57-0fd6-49b2-9d4b-ce23efda5f12 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:41.512767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7b89aecf-2961-47bd-8c3f-bb2d9a5bf8c2 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:41.514254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7b89aecf-2961-47bd-8c3f-bb2d9a5bf8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:41.518570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 890/1799] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:41 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:41.523009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a5a57-0fd6-49b2-9d4b-ce23efda5f12 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:41.529894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 910/1800] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:41.539806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89aecf-2961-47bd-8c3f-bb2d9a5bf8c2 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:41.541456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7b89aecf-2961-47bd-8c3f-bb2d9a5bf8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:41.542578 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a5a57-0fd6-49b2-9d4b-ce23efda5f12 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:41.549859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1540fe-194b-4f89-b217-148dffa8a8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:41.550711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1540fe-194b-4f89-b217-148dffa8a8c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:41.551039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1540fe-194b-4f89-b217-148dffa8a8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:41.551543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1540fe-194b-4f89-b217-148dffa8a8c1 admin admin] PATH_INFO: `/v3/projects/beabf0655a17461d8f09e76132bdfc36/users/bc2cc0b39fdf476fa45fb845cf50cd8b/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:41.553896 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1540fe-194b-4f89-b217-148dffa8a8c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=beabf0655a17461d8f09e76132bdfc36, user_id=bc2cc0b39fdf476fa45fb845cf50cd8b, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:41.558278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1540fe-194b-4f89-b217-148dffa8a8c1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:41.569982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b89aecf-2961-47bd-8c3f-bb2d9a5bf8c2 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:41.575407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 891/1801] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:41 2022] PUT /identity/v3/projects/beabf0655a17461d8f09e76132bdfc36/users/bc2cc0b39fdf476fa45fb845cf50cd8b/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:41.580736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d429a-6fdc-4eee-9006-ec5bcae27a8b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:41.581024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d429a-6fdc-4eee-9006-ec5bcae27a8b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:41.581300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348d429a-6fdc-4eee-9006-ec5bcae27a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:41.622251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-348d429a-6fdc-4eee-9006-ec5bcae27a8b None None] MFA Rules not processed for user `bc2cc0b39fdf476fa45fb845cf50cd8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:41.623461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-348d429a-6fdc-4eee-9006-ec5bcae27a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:41.644590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a1c0d-e994-4c9b-bb19-b83308f9012e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:41.645442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a1c0d-e994-4c9b-bb19-b83308f9012e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:41.645888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a1c0d-e994-4c9b-bb19-b83308f9012e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:41.646395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a1c0d-e994-4c9b-bb19-b83308f9012e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:41.682276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:41.710689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 892/1802] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:41 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:41.719658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a1c0d-e994-4c9b-bb19-b83308f9012e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:41.721143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4a1c0d-e994-4c9b-bb19-b83308f9012e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:41.727384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d429a-6fdc-4eee-9006-ec5bcae27a8b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:41.728961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-348d429a-6fdc-4eee-9006-ec5bcae27a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:41.732078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 911/1803] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:41.799325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348d429a-6fdc-4eee-9006-ec5bcae27a8b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:41.803749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb990251-213f-4da6-b96c-deb8be69c780 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:41.804671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb990251-213f-4da6-b96c-deb8be69c780 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:41.804958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb990251-213f-4da6-b96c-deb8be69c780 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:41.805236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb990251-213f-4da6-b96c-deb8be69c780 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:41.816672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb990251-213f-4da6-b96c-deb8be69c780 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:41.818657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb990251-213f-4da6-b96c-deb8be69c780 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:41.833778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 893/1804] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:41 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:42.151105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a1c0d-e994-4c9b-bb19-b83308f9012e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:42.153645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7f4a1c0d-e994-4c9b-bb19-b83308f9012e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:42.169655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a1c0d-e994-4c9b-bb19-b83308f9012e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:42.177168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:42.178028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:42.178338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:42.178659 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] PATH_INFO: `/v3/projects/09e7b589f6734adc8ac14644b6888355` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:42.179962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09e7b589f6734adc8ac14644b6888355)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:42.182296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:42.195219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:42.254571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09e7b589f6734adc8ac14644b6888355', 'request_id': 'req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.263080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09e7b589f6734adc8ac14644b6888355', 'request_id': 'req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.269618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09e7b589f6734adc8ac14644b6888355', 'request_id': 'req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.290468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] The token cache is being invalidate because project 09e7b589f6734adc8ac14644b6888355 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:42.290960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.291865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.296732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 912/1805] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:42 2022] DELETE /identity/v3/projects/09e7b589f6734adc8ac14644b6888355 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:42.298318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb990251-213f-4da6-b96c-deb8be69c780 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:42.313793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb990251-213f-4da6-b96c-deb8be69c780 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:42.404409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:42.404409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:42.404409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:42.404409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] PATH_INFO: `/v3/users/f04adee616c041bf8a108a9465cbacae` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:42.406377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:42.407364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f04adee616c041bf8a108a9465cbacae)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:42.409007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:42.411415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:42.428841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9ca4ef-fb86-4b74-a0b2-b4981ed399b9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:42.447453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:42.449624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:42.451552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:42.451909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] PATH_INFO: `/v3/projects/250617554d634a3a95733753fc56512c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:42.453698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] RBAC: Authorizing `identity:delete_project(project_id=250617554d634a3a95733753fc56512c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:42.462173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:42.500159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f04adee616c041bf8a108a9465cbacae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.516620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f04adee616c041bf8a108a9465cbacae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.534907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250617554d634a3a95733753fc56512c', 'request_id': 'req-01fe7e67-d616-4c00-be56-3acf5084878d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.539881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f04adee616c041bf8a108a9465cbacae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.540902 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f04adee616c041bf8a108a9465cbacae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.542468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250617554d634a3a95733753fc56512c', 'request_id': 'req-01fe7e67-d616-4c00-be56-3acf5084878d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.549996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f04adee616c041bf8a108a9465cbacae'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.550508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250617554d634a3a95733753fc56512c', 'request_id': 'req-01fe7e67-d616-4c00-be56-3acf5084878d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.560168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 894/1806] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:42 2022] DELETE /identity/v3/users/f04adee616c041bf8a108a9465cbacae => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:42.575884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] The token cache is being invalidate because project 250617554d634a3a95733753fc56512c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:42.576340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.578520 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.581413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 913/1807] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:42 2022] DELETE /identity/v3/projects/250617554d634a3a95733753fc56512c => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:42.585308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:42.587065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:42.609327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:42.702689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:42.704172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:42.728494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1334fe35-0ed8-4ebc-b5d5-dee5866e4b33 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:42.735415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:42.736539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:42.736909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:42.737886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] PATH_INFO: `/v3/users/9716b1b6d02848d2ae6d530058216786` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:42.738959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] RBAC: Authorizing `identity:delete_user(user_id=9716b1b6d02848d2ae6d530058216786)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:42.741369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:42.803058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fe7e67-d616-4c00-be56-3acf5084878d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:42.811164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9716b1b6d02848d2ae6d530058216786'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.821813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9716b1b6d02848d2ae6d530058216786'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.836907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9716b1b6d02848d2ae6d530058216786'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.837769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9716b1b6d02848d2ae6d530058216786'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.842900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:42.846728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9716b1b6d02848d2ae6d530058216786'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:42.856124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 895/1808] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:42 2022] DELETE /identity/v3/users/9716b1b6d02848d2ae6d530058216786 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:42.863439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21543ae1-f31a-4605-b586-94de1f17ebb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:42.864099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543ae1-f31a-4605-b586-94de1f17ebb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:42.864438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543ae1-f31a-4605-b586-94de1f17ebb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:42.865490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21543ae1-f31a-4605-b586-94de1f17ebb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:42.867806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-21543ae1-f31a-4605-b586-94de1f17ebb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:42.948212 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:42.949819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:42.951519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21543ae1-f31a-4605-b586-94de1f17ebb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:42.953175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21543ae1-f31a-4605-b586-94de1f17ebb3 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:42.981852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 914/1809] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:42 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:43.031994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091c7c6-49dc-4a6d-9fc2-2ba0c84683da admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:43.038249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:43.039940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:43.040400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:43.040739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] PATH_INFO: `/v3/projects/6e308f70145b4bac88524fabdacc47bd` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:43.042069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e308f70145b4bac88524fabdacc47bd)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:43.044615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:43.107856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:43.135269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e308f70145b4bac88524fabdacc47bd', 'request_id': 'req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.152908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e308f70145b4bac88524fabdacc47bd', 'request_id': 'req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.160636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e308f70145b4bac88524fabdacc47bd', 'request_id': 'req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.202917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] The token cache is being invalidate because project 6e308f70145b4bac88524fabdacc47bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:43.204514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.205501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.208195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 896/1810] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:42 2022] DELETE /identity/v3/projects/6e308f70145b4bac88524fabdacc47bd => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:43.212120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21543ae1-f31a-4605-b586-94de1f17ebb3 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:43.224635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21543ae1-f31a-4605-b586-94de1f17ebb3 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:43.296883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:43.297517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:43.297796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:43.298150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] PATH_INFO: `/v3/users/c3670045a3a342e4b5320457c878a539` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:43.299300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3670045a3a342e4b5320457c878a539)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:43.301315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:43.318973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:43.320945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:43.348725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3f1c5-f9e7-4f5b-86ab-e41fc8002fd6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:43.355677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:43.356471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:43.356756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:43.357080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] PATH_INFO: `/v3/projects/a64d84dec39f4257835f17fbbb2bd534` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:43.358359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a64d84dec39f4257835f17fbbb2bd534)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:43.361549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:43.405448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3670045a3a342e4b5320457c878a539'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.405635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3670045a3a342e4b5320457c878a539'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.441126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3670045a3a342e4b5320457c878a539'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.470399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3670045a3a342e4b5320457c878a539'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.474050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f21d8d7e-7671-4569-ac5a-05bb9785ba36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3670045a3a342e4b5320457c878a539'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.494847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a64d84dec39f4257835f17fbbb2bd534', 'request_id': 'req-3ef40220-9f38-4371-8215-0ca9ce757f87'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.511846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a64d84dec39f4257835f17fbbb2bd534', 'request_id': 'req-3ef40220-9f38-4371-8215-0ca9ce757f87'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.515712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 915/1811] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:43 2022] DELETE /identity/v3/users/c3670045a3a342e4b5320457c878a539 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:43.535922 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a64d84dec39f4257835f17fbbb2bd534', 'request_id': 'req-3ef40220-9f38-4371-8215-0ca9ce757f87'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.569148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] The token cache is being invalidate because project a64d84dec39f4257835f17fbbb2bd534 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:43.569456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.570251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:43.572338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 897/1812] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:43 2022] DELETE /identity/v3/projects/a64d84dec39f4257835f17fbbb2bd534 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:43.578279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77c439-40a6-448d-aac0-c4f374680f63 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:43.578614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77c439-40a6-448d-aac0-c4f374680f63 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:43.578900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e77c439-40a6-448d-aac0-c4f374680f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:43.608011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4e77c439-40a6-448d-aac0-c4f374680f63 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:43.609895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4e77c439-40a6-448d-aac0-c4f374680f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:43.615902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:43.721116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:43.722501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 916/1813] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:43.722740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:43.741880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8124a-3428-496f-bdbe-b6c8737da120 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:43.742386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8124a-3428-496f-bdbe-b6c8737da120 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:43.742815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c8124a-3428-496f-bdbe-b6c8737da120 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:43.800984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-77c8124a-3428-496f-bdbe-b6c8737da120 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:43.806217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-77c8124a-3428-496f-bdbe-b6c8737da120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:43.819802 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef40220-9f38-4371-8215-0ca9ce757f87 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:43.828556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 917/1814] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:43.832864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8124a-3428-496f-bdbe-b6c8737da120 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:43.835867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:43.836836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:43.837487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:43.837900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] PATH_INFO: `/v3/projects/879272babecc4dfda0b9bffab5cb5432` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:43.839677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] RBAC: Authorizing `identity:delete_project(project_id=879272babecc4dfda0b9bffab5cb5432)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:43.855086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c8124a-3428-496f-bdbe-b6c8737da120 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:43.855860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:43.869407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467458d8-83b1-43d3-9fca-e989d4ef033e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:43.870044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467458d8-83b1-43d3-9fca-e989d4ef033e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:43.870507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467458d8-83b1-43d3-9fca-e989d4ef033e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:43.870786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467458d8-83b1-43d3-9fca-e989d4ef033e admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:43.871615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467458d8-83b1-43d3-9fca-e989d4ef033e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:43.871921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467458d8-83b1-43d3-9fca-e989d4ef033e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:43.874170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467458d8-83b1-43d3-9fca-e989d4ef033e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:43.919774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 918/1815] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:43.923958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3b04-40b3-4a8c-baf4-a4d701f03855 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:43.930023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3b04-40b3-4a8c-baf4-a4d701f03855 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:43.930337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be3b04-40b3-4a8c-baf4-a4d701f03855 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:43.932289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 919/1816] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:01:43 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:01:43.932945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3b04-40b3-4a8c-baf4-a4d701f03855 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:43.965307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be3b04-40b3-4a8c-baf4-a4d701f03855 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.006328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:44.011419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '879272babecc4dfda0b9bffab5cb5432', 'request_id': 'req-8c0007d5-3479-4c14-ae5f-edc624968171'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.035058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '879272babecc4dfda0b9bffab5cb5432', 'request_id': 'req-8c0007d5-3479-4c14-ae5f-edc624968171'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.058155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '879272babecc4dfda0b9bffab5cb5432', 'request_id': 'req-8c0007d5-3479-4c14-ae5f-edc624968171'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.079134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db84d6d-a98f-43c2-ba51-852273b0dbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.080998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db84d6d-a98f-43c2-ba51-852273b0dbd6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.082619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db84d6d-a98f-43c2-ba51-852273b0dbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.083287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db84d6d-a98f-43c2-ba51-852273b0dbd6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.087472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db84d6d-a98f-43c2-ba51-852273b0dbd6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.090986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] The token cache is being invalidate because project 879272babecc4dfda0b9bffab5cb5432 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:44.091396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.092856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8c0007d5-3479-4c14-ae5f-edc624968171 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.094019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db84d6d-a98f-43c2-ba51-852273b0dbd6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.094942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 898/1817] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:43 2022] DELETE /identity/v3/projects/879272babecc4dfda0b9bffab5cb5432 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:44.096911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88715879-1ee2-4a6a-aef0-3bfe4087ffaf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.097501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88715879-1ee2-4a6a-aef0-3bfe4087ffaf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.097731 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88715879-1ee2-4a6a-aef0-3bfe4087ffaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.127093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-88715879-1ee2-4a6a-aef0-3bfe4087ffaf None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:44.128573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-88715879-1ee2-4a6a-aef0-3bfe4087ffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:44.139132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 920/1818] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:43 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:44.140840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61665b2-ab9f-44a7-b12c-3fd1e54c07a4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.141268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61665b2-ab9f-44a7-b12c-3fd1e54c07a4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.141676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61665b2-ab9f-44a7-b12c-3fd1e54c07a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.180295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d61665b2-ab9f-44a7-b12c-3fd1e54c07a4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:44.182591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d61665b2-ab9f-44a7-b12c-3fd1e54c07a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:44.241561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 899/1819] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:01:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:44.242551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88715879-1ee2-4a6a-aef0-3bfe4087ffaf None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.243956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-88715879-1ee2-4a6a-aef0-3bfe4087ffaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:44.264115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88715879-1ee2-4a6a-aef0-3bfe4087ffaf None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.273663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 921/1820] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:44.275558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8b0a8-d0d7-43ab-a6a2-27061916cecc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.275852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8b0a8-d0d7-43ab-a6a2-27061916cecc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.276126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d8b0a8-d0d7-43ab-a6a2-27061916cecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.278543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba81aa-f9be-41d7-81fe-402907764b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.280187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba81aa-f9be-41d7-81fe-402907764b00 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.280639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba81aa-f9be-41d7-81fe-402907764b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.281015 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ba81aa-f9be-41d7-81fe-402907764b00 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.282263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba81aa-f9be-41d7-81fe-402907764b00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.284437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ba81aa-f9be-41d7-81fe-402907764b00 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.299249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c0d8b0a8-d0d7-43ab-a6a2-27061916cecc None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:44.300969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c0d8b0a8-d0d7-43ab-a6a2-27061916cecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:44.313389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 922/1821] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:44.318421 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8b0a8-d0d7-43ab-a6a2-27061916cecc None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.323095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 900/1822] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:44 2022] POST /identity/v3/users => generated 291 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:44.327134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba81aa-f9be-41d7-81fe-402907764b00 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.331589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d8b0a8-d0d7-43ab-a6a2-27061916cecc None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.338071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ffe09-96ff-4f7b-ab5f-77c0e936b6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.338980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ffe09-96ff-4f7b-ab5f-77c0e936b6bb admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.339326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ffe09-96ff-4f7b-ab5f-77c0e936b6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.339656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8ffe09-96ff-4f7b-ab5f-77c0e936b6bb admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.340464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ffe09-96ff-4f7b-ab5f-77c0e936b6bb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:44.340830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ffe09-96ff-4f7b-ab5f-77c0e936b6bb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.341378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ba81aa-f9be-41d7-81fe-402907764b00 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.344081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8ffe09-96ff-4f7b-ab5f-77c0e936b6bb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.347330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce006ce-052a-40f2-aecc-acfab7bf36e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.348035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce006ce-052a-40f2-aecc-acfab7bf36e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.348584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce006ce-052a-40f2-aecc-acfab7bf36e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.348742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce006ce-052a-40f2-aecc-acfab7bf36e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.349619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce006ce-052a-40f2-aecc-acfab7bf36e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.349797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:44.351692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce006ce-052a-40f2-aecc-acfab7bf36e7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.372752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 923/1823] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:44.373727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ffe09-96ff-4f7b-ab5f-77c0e936b6bb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.378475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 901/1824] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:44.379961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce006ce-052a-40f2-aecc-acfab7bf36e7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.388208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8ffe09-96ff-4f7b-ab5f-77c0e936b6bb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.392613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce006ce-052a-40f2-aecc-acfab7bf36e7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.393668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa1dd0-2c15-4ab8-89a0-19ce6d46a79a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.394413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa1dd0-2c15-4ab8-89a0-19ce6d46a79a admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.394752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa1dd0-2c15-4ab8-89a0-19ce6d46a79a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.395088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaa1dd0-2c15-4ab8-89a0-19ce6d46a79a admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.396233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa1dd0-2c15-4ab8-89a0-19ce6d46a79a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.398289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaa1dd0-2c15-4ab8-89a0-19ce6d46a79a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.399908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54335656-d687-4946-986a-cbf1c795e25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.401042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54335656-d687-4946-986a-cbf1c795e25c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.401607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54335656-d687-4946-986a-cbf1c795e25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.402101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54335656-d687-4946-986a-cbf1c795e25c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.402917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54335656-d687-4946-986a-cbf1c795e25c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.405543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54335656-d687-4946-986a-cbf1c795e25c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.418224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 902/1825] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:44.422759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54335656-d687-4946-986a-cbf1c795e25c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.433834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54335656-d687-4946-986a-cbf1c795e25c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.437386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 924/1826] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:44 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:44.438485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa1dd0-2c15-4ab8-89a0-19ce6d46a79a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.515329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c814e14-72d6-41b9-b142-0475f8017e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.516024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c814e14-72d6-41b9-b142-0475f8017e9b admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.516356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c814e14-72d6-41b9-b142-0475f8017e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.516907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c814e14-72d6-41b9-b142-0475f8017e9b admin admin] PATH_INFO: `/v3/projects/c6cf225f61ed4f0bbcdb9533d608a947/users/3cebc083b9f3467c996b97492232a4b4/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.539805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c814e14-72d6-41b9-b142-0475f8017e9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6cf225f61ed4f0bbcdb9533d608a947, user_id=3cebc083b9f3467c996b97492232a4b4, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.543620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c814e14-72d6-41b9-b142-0475f8017e9b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.543765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaa1dd0-2c15-4ab8-89a0-19ce6d46a79a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.553081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b187f6-e1ca-4563-88bf-1315d567cfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.553740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b187f6-e1ca-4563-88bf-1315d567cfe8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.555310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b187f6-e1ca-4563-88bf-1315d567cfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.555580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b187f6-e1ca-4563-88bf-1315d567cfe8 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.558818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b187f6-e1ca-4563-88bf-1315d567cfe8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.561456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b187f6-e1ca-4563-88bf-1315d567cfe8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.574403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 903/1827] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:44 2022] PUT /identity/v3/projects/c6cf225f61ed4f0bbcdb9533d608a947/users/3cebc083b9f3467c996b97492232a4b4/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:44.578133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f7d92-a9a9-4426-befe-951073af159d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.578436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f7d92-a9a9-4426-befe-951073af159d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.580203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f7d92-a9a9-4426-befe-951073af159d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.613938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 925/1828] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:44 2022] POST /identity/v3/users => generated 293 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:44.625834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=3,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:44.633508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b187f6-e1ca-4563-88bf-1315d567cfe8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.647658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-126f7d92-a9a9-4426-befe-951073af159d None None] MFA Rules not processed for user `3cebc083b9f3467c996b97492232a4b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:44.652026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-126f7d92-a9a9-4426-befe-951073af159d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:44.663849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b187f6-e1ca-4563-88bf-1315d567cfe8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.751769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 904/1829] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:44 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:44.751769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f7d92-a9a9-4426-befe-951073af159d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.751769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-126f7d92-a9a9-4426-befe-951073af159d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:44.778918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3dde3-7240-4923-81e0-914181fb8c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.779979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3dde3-7240-4923-81e0-914181fb8c31 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.780255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3dde3-7240-4923-81e0-914181fb8c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.780533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9c3dde3-7240-4923-81e0-914181fb8c31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.781324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3dde3-7240-4923-81e0-914181fb8c31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.782979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9c3dde3-7240-4923-81e0-914181fb8c31 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.802322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 926/1830] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:44.802322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3dde3-7240-4923-81e0-914181fb8c31 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.802322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9c3dde3-7240-4923-81e0-914181fb8c31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:44.824182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9c3dde3-7240-4923-81e0-914181fb8c31 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.829917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.831926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.831926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.831926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] PATH_INFO: `/v3/projects/fe1b3a69ea2b42788ac936f726149b32` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.832315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1b3a69ea2b42788ac936f726149b32)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.834503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.838655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f7d92-a9a9-4426-befe-951073af159d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.844391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305770e-5669-41ca-bf13-f663301ca6dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:44.845133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305770e-5669-41ca-bf13-f663301ca6dc service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:44.845417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305770e-5669-41ca-bf13-f663301ca6dc service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:44.845689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e305770e-5669-41ca-bf13-f663301ca6dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:44.858204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305770e-5669-41ca-bf13-f663301ca6dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:44.859999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e305770e-5669-41ca-bf13-f663301ca6dc service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:44.883661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 905/1831] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:44.884773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305770e-5669-41ca-bf13-f663301ca6dc service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.911818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1b3a69ea2b42788ac936f726149b32', 'request_id': 'req-28edabdb-c7f3-4457-8a20-cadb57ce437c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.917014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e305770e-5669-41ca-bf13-f663301ca6dc service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:44.927928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1b3a69ea2b42788ac936f726149b32', 'request_id': 'req-28edabdb-c7f3-4457-8a20-cadb57ce437c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.931888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:INSERT=4,keystone:DELETE=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:44.955707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1b3a69ea2b42788ac936f726149b32', 'request_id': 'req-28edabdb-c7f3-4457-8a20-cadb57ce437c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.984125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] The token cache is being invalidate because project fe1b3a69ea2b42788ac936f726149b32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:44.985104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.987056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:44.988897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 927/1832] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:44 2022] DELETE /identity/v3/projects/fe1b3a69ea2b42788ac936f726149b32 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:44.993936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:44.995595 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:45.036206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062aada1-6e28-4145-b900-4a2fae773022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.037705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062aada1-6e28-4145-b900-4a2fae773022 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.038051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062aada1-6e28-4145-b900-4a2fae773022 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.038422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062aada1-6e28-4145-b900-4a2fae773022 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.039137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062aada1-6e28-4145-b900-4a2fae773022 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.041897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062aada1-6e28-4145-b900-4a2fae773022 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.093131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 906/1833] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:44 2022] POST /identity/v3/projects => generated 381 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:45.095907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcce63-58e2-4415-a54a-62a47c27c127 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.096284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcce63-58e2-4415-a54a-62a47c27c127 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.096632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fcce63-58e2-4415-a54a-62a47c27c127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.100071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28edabdb-c7f3-4457-8a20-cadb57ce437c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.124991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-37fcce63-58e2-4415-a54a-62a47c27c127 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:45.126867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-37fcce63-58e2-4415-a54a-62a47c27c127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:45.139681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:45.176036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc07883-1c35-46fa-8196-6eac6e40454a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.176683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc07883-1c35-46fa-8196-6eac6e40454a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.178416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc07883-1c35-46fa-8196-6eac6e40454a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.178416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc07883-1c35-46fa-8196-6eac6e40454a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.179101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc07883-1c35-46fa-8196-6eac6e40454a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.181236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc07883-1c35-46fa-8196-6eac6e40454a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.193261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 928/1834] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:44 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:45.194224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc07883-1c35-46fa-8196-6eac6e40454a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.196283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4fc07883-1c35-46fa-8196-6eac6e40454a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:45.215302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc07883-1c35-46fa-8196-6eac6e40454a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.226234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760ba425-c3a5-44e4-add4-bf38013e7c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.227802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ba425-c3a5-44e4-add4-bf38013e7c61 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.229643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ba425-c3a5-44e4-add4-bf38013e7c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.229932 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760ba425-c3a5-44e4-add4-bf38013e7c61 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.231794 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ba425-c3a5-44e4-add4-bf38013e7c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.234883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760ba425-c3a5-44e4-add4-bf38013e7c61 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.249036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 907/1835] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:45.250388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcce63-58e2-4415-a54a-62a47c27c127 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.266887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fcce63-58e2-4415-a54a-62a47c27c127 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.268950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 929/1836] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:45 2022] POST /identity/v3/users => generated 297 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:45.271463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f238c-1b42-4f2d-80ab-452ccc66f278 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.272556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f238c-1b42-4f2d-80ab-452ccc66f278 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.273048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3f238c-1b42-4f2d-80ab-452ccc66f278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.278162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c25a0-173e-4f95-a38e-ee7718b68be3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.278909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c25a0-173e-4f95-a38e-ee7718b68be3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.279224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c25a0-173e-4f95-a38e-ee7718b68be3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.279441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50c25a0-173e-4f95-a38e-ee7718b68be3 admin admin] PATH_INFO: `/v3/projects/a762166bdca741418d9f4ccf7b3aa231/users/675e3d10ef2244d494a0135053e529fe/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.305439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c25a0-173e-4f95-a38e-ee7718b68be3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a762166bdca741418d9f4ccf7b3aa231, user_id=675e3d10ef2244d494a0135053e529fe, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.311045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a50c25a0-173e-4f95-a38e-ee7718b68be3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.313014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-db3f238c-1b42-4f2d-80ab-452ccc66f278 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:45.314553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db3f238c-1b42-4f2d-80ab-452ccc66f278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:45.324217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 930/1837] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:45.325379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f238c-1b42-4f2d-80ab-452ccc66f278 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.330741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 908/1838] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:45 2022] PUT /identity/v3/projects/a762166bdca741418d9f4ccf7b3aa231/users/675e3d10ef2244d494a0135053e529fe/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:45.332322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c25a0-173e-4f95-a38e-ee7718b68be3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.336261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3f238c-1b42-4f2d-80ab-452ccc66f278 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.403519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22243849-9a52-4228-966f-027a4300b154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.404287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22243849-9a52-4228-966f-027a4300b154 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.404490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22243849-9a52-4228-966f-027a4300b154 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.404758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22243849-9a52-4228-966f-027a4300b154 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.405586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22243849-9a52-4228-966f-027a4300b154 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.407865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22243849-9a52-4228-966f-027a4300b154 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.410224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:45.412769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50c25a0-173e-4f95-a38e-ee7718b68be3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.419819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9c793-a81f-468c-bd89-9831cc543e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.421231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9c793-a81f-468c-bd89-9831cc543e1c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.421677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9c793-a81f-468c-bd89-9831cc543e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.421955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e9c793-a81f-468c-bd89-9831cc543e1c admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.423106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9c793-a81f-468c-bd89-9831cc543e1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:45.423106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9c793-a81f-468c-bd89-9831cc543e1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.423988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 931/1839] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:45.424870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e9c793-a81f-468c-bd89-9831cc543e1c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.426063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a16d1f-91fa-44e8-bcf8-ad1e7d0b0bf4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.426545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a16d1f-91fa-44e8-bcf8-ad1e7d0b0bf4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.427207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a16d1f-91fa-44e8-bcf8-ad1e7d0b0bf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.444053 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 909/1840] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:45.444053 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9c793-a81f-468c-bd89-9831cc543e1c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.457616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e9c793-a81f-468c-bd89-9831cc543e1c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.459035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d7a16d1f-91fa-44e8-bcf8-ad1e7d0b0bf4 None None] MFA Rules not processed for user `675e3d10ef2244d494a0135053e529fe`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:45.460315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d7a16d1f-91fa-44e8-bcf8-ad1e7d0b0bf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:45.462966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7841405-26f0-4221-be05-90c7eb765bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.463647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7841405-26f0-4221-be05-90c7eb765bc8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.463925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7841405-26f0-4221-be05-90c7eb765bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.464201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7841405-26f0-4221-be05-90c7eb765bc8 admin admin] PATH_INFO: `/v3/projects/56bdc0a62ae0474e80f5d02fe8d70b40/users/17d56e5731664f82b6e53693eec68c3c/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.480561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7841405-26f0-4221-be05-90c7eb765bc8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bdc0a62ae0474e80f5d02fe8d70b40, user_id=17d56e5731664f82b6e53693eec68c3c, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.483291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7841405-26f0-4221-be05-90c7eb765bc8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.503211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 910/1841] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:45 2022] PUT /identity/v3/projects/56bdc0a62ae0474e80f5d02fe8d70b40/users/17d56e5731664f82b6e53693eec68c3c/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:45.504416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7841405-26f0-4221-be05-90c7eb765bc8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.538222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 932/1842] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:45 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:45.540258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a16d1f-91fa-44e8-bcf8-ad1e7d0b0bf4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.557203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a16d1f-91fa-44e8-bcf8-ad1e7d0b0bf4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.580649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:45.585788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7841405-26f0-4221-be05-90c7eb765bc8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.602940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12f869-9be0-4943-a556-2ebfab3db6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.603629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12f869-9be0-4943-a556-2ebfab3db6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.603961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12f869-9be0-4943-a556-2ebfab3db6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.604243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e12f869-9be0-4943-a556-2ebfab3db6c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.605796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12f869-9be0-4943-a556-2ebfab3db6c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.607643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e12f869-9be0-4943-a556-2ebfab3db6c6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.642296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa254b-ae7e-44de-bc8b-9fe4d156eaff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.642928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa254b-ae7e-44de-bc8b-9fe4d156eaff admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.643211 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa254b-ae7e-44de-bc8b-9fe4d156eaff admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.643487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa254b-ae7e-44de-bc8b-9fe4d156eaff admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.643799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 911/1843] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:45 2022] POST /identity/v3/projects => generated 369 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:45.644134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa254b-ae7e-44de-bc8b-9fe4d156eaff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.645779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa254b-ae7e-44de-bc8b-9fe4d156eaff admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.647168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12f869-9be0-4943-a556-2ebfab3db6c6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.648872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6e12f869-9be0-4943-a556-2ebfab3db6c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:45.660522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 933/1844] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:45.660522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa254b-ae7e-44de-bc8b-9fe4d156eaff admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.669315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa254b-ae7e-44de-bc8b-9fe4d156eaff admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.732072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e12f869-9be0-4943-a556-2ebfab3db6c6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.739161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07a6ed-9201-4df3-9431-5c3f876fd23b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.739880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07a6ed-9201-4df3-9431-5c3f876fd23b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.740171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07a6ed-9201-4df3-9431-5c3f876fd23b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.740546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e07a6ed-9201-4df3-9431-5c3f876fd23b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.752267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720ae9-eabc-4061-933e-f6e99e7d4028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.753179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720ae9-eabc-4061-933e-f6e99e7d4028 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.753472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720ae9-eabc-4061-933e-f6e99e7d4028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.753728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53720ae9-eabc-4061-933e-f6e99e7d4028 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.754480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720ae9-eabc-4061-933e-f6e99e7d4028 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.757219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53720ae9-eabc-4061-933e-f6e99e7d4028 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.785765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 934/1845] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:45 2022] POST /identity/v3/users => generated 285 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:45.785765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720ae9-eabc-4061-933e-f6e99e7d4028 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.809215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53720ae9-eabc-4061-933e-f6e99e7d4028 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.814779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea62c05c-5e14-4780-8d01-0aa2bf6e47a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.815526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62c05c-5e14-4780-8d01-0aa2bf6e47a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.815873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62c05c-5e14-4780-8d01-0aa2bf6e47a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.816145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea62c05c-5e14-4780-8d01-0aa2bf6e47a8 admin admin] PATH_INFO: `/v3/projects/56bdc0a62ae0474e80f5d02fe8d70b40/users/17d56e5731664f82b6e53693eec68c3c/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.816971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07a6ed-9201-4df3-9431-5c3f876fd23b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.818867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e07a6ed-9201-4df3-9431-5c3f876fd23b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.819209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62c05c-5e14-4780-8d01-0aa2bf6e47a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56bdc0a62ae0474e80f5d02fe8d70b40, user_id=17d56e5731664f82b6e53693eec68c3c, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.821875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea62c05c-5e14-4780-8d01-0aa2bf6e47a8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.835907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 912/1846] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:45 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:45.836608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:45.836998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07a6ed-9201-4df3-9431-5c3f876fd23b service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:45.846340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 935/1847] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:45 2022] PUT /identity/v3/projects/56bdc0a62ae0474e80f5d02fe8d70b40/users/17d56e5731664f82b6e53693eec68c3c/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:45.858186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e07a6ed-9201-4df3-9431-5c3f876fd23b service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:45.859680 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dedb2-0852-4f71-bf96-3baea76dd348 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.860077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dedb2-0852-4f71-bf96-3baea76dd348 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.860604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838dedb2-0852-4f71-bf96-3baea76dd348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.914670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-838dedb2-0852-4f71-bf96-3baea76dd348 None None] MFA Rules not processed for user `17d56e5731664f82b6e53693eec68c3c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:45.916384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-838dedb2-0852-4f71-bf96-3baea76dd348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:45.966942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8579d5-38e3-473a-beab-e979980fb745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:45.967908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8579d5-38e3-473a-beab-e979980fb745 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:45.968361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8579d5-38e3-473a-beab-e979980fb745 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:45.968838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8579d5-38e3-473a-beab-e979980fb745 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:45.969686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8579d5-38e3-473a-beab-e979980fb745 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:45.972620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8579d5-38e3-473a-beab-e979980fb745 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:45.987672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 913/1848] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:45 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:45.989669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8579d5-38e3-473a-beab-e979980fb745 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:46.005267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8579d5-38e3-473a-beab-e979980fb745 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:46.015564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:46.016240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.016519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.016795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] PATH_INFO: `/v3/users/675e3d10ef2244d494a0135053e529fe` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.018875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=675e3d10ef2244d494a0135053e529fe)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:46.021731 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:46.042729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 936/1849] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:45 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:46.043621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dedb2-0852-4f71-bf96-3baea76dd348 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:46.063292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838dedb2-0852-4f71-bf96-3baea76dd348 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:46.069096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8e1d9-3772-4e5d-93aa-4c32a7218626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:46.069741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e1d9-3772-4e5d-93aa-4c32a7218626 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.070025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e1d9-3772-4e5d-93aa-4c32a7218626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.070494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8e1d9-3772-4e5d-93aa-4c32a7218626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.071458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8e1d9-3772-4e5d-93aa-4c32a7218626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:46.074256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8e1d9-3772-4e5d-93aa-4c32a7218626 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:46.090113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 937/1850] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:46.095482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8e1d9-3772-4e5d-93aa-4c32a7218626 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:46.098467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-59a8e1d9-3772-4e5d-93aa-4c32a7218626 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:46.106491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:46.150272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '675e3d10ef2244d494a0135053e529fe'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:46.172298 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '675e3d10ef2244d494a0135053e529fe'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:46.192093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '675e3d10ef2244d494a0135053e529fe'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:46.193527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '675e3d10ef2244d494a0135053e529fe'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:46.202813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '675e3d10ef2244d494a0135053e529fe'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:46.210913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 914/1851] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:45 2022] DELETE /identity/v3/users/675e3d10ef2244d494a0135053e529fe => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:46.211967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:46.214123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:46.217068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8e1d9-3772-4e5d-93aa-4c32a7218626 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:46.292957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5212e098-0b5c-4822-a2ca-f879de811df9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:46.293515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212e098-0b5c-4822-a2ca-f879de811df9 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.293798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212e098-0b5c-4822-a2ca-f879de811df9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.294072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5212e098-0b5c-4822-a2ca-f879de811df9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.296346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5212e098-0b5c-4822-a2ca-f879de811df9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:46.301341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55f2a8-3715-414e-a6d6-7e9143e88df8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:46.309382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d653f2ed-9e93-4087-9785-ad413f0f77c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:46.322780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653f2ed-9e93-4087-9785-ad413f0f77c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.324413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653f2ed-9e93-4087-9785-ad413f0f77c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.324845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d653f2ed-9e93-4087-9785-ad413f0f77c7 admin admin] PATH_INFO: `/v3/projects/52fa82c56bf44a8badd6d1697ec5b2bc/users/3391943c06c6409eb8262038b78f5949/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.353418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653f2ed-9e93-4087-9785-ad413f0f77c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=52fa82c56bf44a8badd6d1697ec5b2bc, user_id=3391943c06c6409eb8262038b78f5949, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:46.357112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d653f2ed-9e93-4087-9785-ad413f0f77c7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:46.369357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:46.380971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 915/1852] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:46 2022] PUT /identity/v3/projects/52fa82c56bf44a8badd6d1697ec5b2bc/users/3391943c06c6409eb8262038b78f5949/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:46.384098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e514f6-113f-49ad-9fd8-e06324bb6e47 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.384529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e514f6-113f-49ad-9fd8-e06324bb6e47 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.385036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e514f6-113f-49ad-9fd8-e06324bb6e47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.422791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212e098-0b5c-4822-a2ca-f879de811df9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:46.424256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5212e098-0b5c-4822-a2ca-f879de811df9 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:46.426010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-60e514f6-113f-49ad-9fd8-e06324bb6e47 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:46.428421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60e514f6-113f-49ad-9fd8-e06324bb6e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:46.443467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 938/1853] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:46 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:46.456433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526e6cd-54c4-40be-905b-a0be279d775c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.456719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526e6cd-54c4-40be-905b-a0be279d775c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.456997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c526e6cd-54c4-40be-905b-a0be279d775c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.529124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c526e6cd-54c4-40be-905b-a0be279d775c None None] MFA Rules not processed for user `3391943c06c6409eb8262038b78f5949`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:46.531125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c526e6cd-54c4-40be-905b-a0be279d775c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:46.535816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 916/1854] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:46.551261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e514f6-113f-49ad-9fd8-e06324bb6e47 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:46.554274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-60e514f6-113f-49ad-9fd8-e06324bb6e47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:46.603999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:46.642977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e514f6-113f-49ad-9fd8-e06324bb6e47 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:46.647686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ce93e-d50e-4f10-b9bd-a677fd2a04c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:46.648322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ce93e-d50e-4f10-b9bd-a677fd2a04c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.648599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ce93e-d50e-4f10-b9bd-a677fd2a04c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.648870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8ce93e-d50e-4f10-b9bd-a677fd2a04c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.659098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ce93e-d50e-4f10-b9bd-a677fd2a04c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:46.660553 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8ce93e-d50e-4f10-b9bd-a677fd2a04c0 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:46.673330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 917/1855] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:46.674033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ce93e-d50e-4f10-b9bd-a677fd2a04c0 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:46.676005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0d8ce93e-d50e-4f10-b9bd-a677fd2a04c0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:46.680326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 939/1856] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:46.685018 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526e6cd-54c4-40be-905b-a0be279d775c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:46.697512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c526e6cd-54c4-40be-905b-a0be279d775c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:46.702582 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5032eb-86a1-463e-9d08-e63d3ea4866c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:46.703333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5032eb-86a1-463e-9d08-e63d3ea4866c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.703680 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5032eb-86a1-463e-9d08-e63d3ea4866c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.704023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5032eb-86a1-463e-9d08-e63d3ea4866c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.704745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5032eb-86a1-463e-9d08-e63d3ea4866c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:46.706842 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5032eb-86a1-463e-9d08-e63d3ea4866c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:46.755075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 940/1857] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:46 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:46.758386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5032eb-86a1-463e-9d08-e63d3ea4866c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:46.765342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8ce93e-d50e-4f10-b9bd-a677fd2a04c0 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:46.857266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9822b-6ac3-4e61-afad-33060df22e59 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:46.859377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9822b-6ac3-4e61-afad-33060df22e59 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.859377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9822b-6ac3-4e61-afad-33060df22e59 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.859377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b9822b-6ac3-4e61-afad-33060df22e59 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.862137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-57b9822b-6ac3-4e61-afad-33060df22e59 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:46.913823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5032eb-86a1-463e-9d08-e63d3ea4866c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:46.926765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d7b9-7a02-4ee2-a2d1-d5e379a31721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:46.928206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d7b9-7a02-4ee2-a2d1-d5e379a31721 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:46.928553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d7b9-7a02-4ee2-a2d1-d5e379a31721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:46.928897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed5d7b9-7a02-4ee2-a2d1-d5e379a31721 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:46.930011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5d7b9-7a02-4ee2-a2d1-d5e379a31721 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:46.932210 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed5d7b9-7a02-4ee2-a2d1-d5e379a31721 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:46.966324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9822b-6ac3-4e61-afad-33060df22e59 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:46.968365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b9822b-6ac3-4e61-afad-33060df22e59 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:46.986600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 918/1858] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:46 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:46.986921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 941/1859] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:46 2022] POST /identity/v3/users => generated 291 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:46.990734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d7b9-7a02-4ee2-a2d1-d5e379a31721 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.017507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed5d7b9-7a02-4ee2-a2d1-d5e379a31721 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.024379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc6b84-81e0-4ed8-b10e-89b9227513d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.025008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc6b84-81e0-4ed8-b10e-89b9227513d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.025285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc6b84-81e0-4ed8-b10e-89b9227513d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.025564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabc6b84-81e0-4ed8-b10e-89b9227513d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.026402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc6b84-81e0-4ed8-b10e-89b9227513d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:47.028057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabc6b84-81e0-4ed8-b10e-89b9227513d6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:47.044250 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 942/1860] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:46 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:01:47.048457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9822b-6ac3-4e61-afad-33060df22e59 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.062239 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b9822b-6ac3-4e61-afad-33060df22e59 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.067178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92c00d-2c4e-4201-bc1e-a246e1e588b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.068591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92c00d-2c4e-4201-bc1e-a246e1e588b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.068878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92c00d-2c4e-4201-bc1e-a246e1e588b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.069157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c92c00d-2c4e-4201-bc1e-a246e1e588b3 admin admin] PATH_INFO: `/v3/projects/c30ac5fa774948be87174d2ce4e05bdf/users/76cf5ed34d5045a6af07bb96b5016d8f/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.090580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92c00d-2c4e-4201-bc1e-a246e1e588b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30ac5fa774948be87174d2ce4e05bdf, user_id=76cf5ed34d5045a6af07bb96b5016d8f, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:47.093810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c92c00d-2c4e-4201-bc1e-a246e1e588b3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:47.112446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc6b84-81e0-4ed8-b10e-89b9227513d6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.114376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dabc6b84-81e0-4ed8-b10e-89b9227513d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.118897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 919/1861] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:47 2022] PUT /identity/v3/projects/c30ac5fa774948be87174d2ce4e05bdf/users/76cf5ed34d5045a6af07bb96b5016d8f/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:47.123185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92c00d-2c4e-4201-bc1e-a246e1e588b3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.134191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabc6b84-81e0-4ed8-b10e-89b9227513d6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.183211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:47.203704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:47.255458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.256683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.257144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.257609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] PATH_INFO: `/v3/projects/a762166bdca741418d9f4ccf7b3aa231` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.259067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a762166bdca741418d9f4ccf7b3aa231)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:47.261379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:47.262220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c92c00d-2c4e-4201-bc1e-a246e1e588b3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.269916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920864e-a058-4d65-ab0e-407ce56292cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.271716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920864e-a058-4d65-ab0e-407ce56292cf admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.272072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920864e-a058-4d65-ab0e-407ce56292cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.272798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7920864e-a058-4d65-ab0e-407ce56292cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.273463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920864e-a058-4d65-ab0e-407ce56292cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:47.275715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7920864e-a058-4d65-ab0e-407ce56292cf admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:47.289712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 920/1862] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:47 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:47.293735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920864e-a058-4d65-ab0e-407ce56292cf admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.313024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7920864e-a058-4d65-ab0e-407ce56292cf admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.324840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d27452-f66b-4aba-a9aa-e39302109b63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.324840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d27452-f66b-4aba-a9aa-e39302109b63 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.324840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d27452-f66b-4aba-a9aa-e39302109b63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.324840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d27452-f66b-4aba-a9aa-e39302109b63 admin admin] PATH_INFO: `/v3/projects/c30ac5fa774948be87174d2ce4e05bdf/users/76cf5ed34d5045a6af07bb96b5016d8f/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.325318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d27452-f66b-4aba-a9aa-e39302109b63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30ac5fa774948be87174d2ce4e05bdf, user_id=76cf5ed34d5045a6af07bb96b5016d8f, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:47.328299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d27452-f66b-4aba-a9aa-e39302109b63 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:47.347258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a762166bdca741418d9f4ccf7b3aa231', 'request_id': 'req-4fff62a7-8ba5-4beb-9990-849b2e6143d9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.353265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 921/1863] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:47 2022] PUT /identity/v3/projects/c30ac5fa774948be87174d2ce4e05bdf/users/76cf5ed34d5045a6af07bb96b5016d8f/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:47.355958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a762166bdca741418d9f4ccf7b3aa231', 'request_id': 'req-4fff62a7-8ba5-4beb-9990-849b2e6143d9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.362741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a762166bdca741418d9f4ccf7b3aa231', 'request_id': 'req-4fff62a7-8ba5-4beb-9990-849b2e6143d9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.365160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed54a-7073-48d0-8bae-0fe0cacd9b74 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.366504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed54a-7073-48d0-8bae-0fe0cacd9b74 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.367474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580ed54a-7073-48d0-8bae-0fe0cacd9b74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.387640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] The token cache is being invalidate because project a762166bdca741418d9f4ccf7b3aa231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:47.388166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.388601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4fff62a7-8ba5-4beb-9990-849b2e6143d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.393502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 943/1864] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:47 2022] DELETE /identity/v3/projects/a762166bdca741418d9f4ccf7b3aa231 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:47.395270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223194-4e58-4d9b-bbcc-1883e3e9e378 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.396009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223194-4e58-4d9b-bbcc-1883e3e9e378 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.396294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b223194-4e58-4d9b-bbcc-1883e3e9e378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.415250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-580ed54a-7073-48d0-8bae-0fe0cacd9b74 None None] MFA Rules not processed for user `76cf5ed34d5045a6af07bb96b5016d8f`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:47.416891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-580ed54a-7073-48d0-8bae-0fe0cacd9b74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.422595 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8b223194-4e58-4d9b-bbcc-1883e3e9e378 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:47.424364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8b223194-4e58-4d9b-bbcc-1883e3e9e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.496973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 944/1865] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:47.497862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223194-4e58-4d9b-bbcc-1883e3e9e378 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.499208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8b223194-4e58-4d9b-bbcc-1883e3e9e378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.511813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b223194-4e58-4d9b-bbcc-1883e3e9e378 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.514646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 922/1866] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:47 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:47.515645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e00a-ee69-4aad-8cf4-a357c59b7366 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.515929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e00a-ee69-4aad-8cf4-a357c59b7366 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.516258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb4e00a-ee69-4aad-8cf4-a357c59b7366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.517635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.518420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.518771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.519200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] PATH_INFO: `/v3/users/3cebc083b9f3467c996b97492232a4b4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.520404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cebc083b9f3467c996b97492232a4b4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:47.523152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:47.546841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-deb4e00a-ee69-4aad-8cf4-a357c59b7366 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:47.548063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-deb4e00a-ee69-4aad-8cf4-a357c59b7366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.562164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 923/1867] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:47.562164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4e00a-ee69-4aad-8cf4-a357c59b7366 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.562164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-deb4e00a-ee69-4aad-8cf4-a357c59b7366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.594886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cebc083b9f3467c996b97492232a4b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.596016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cebc083b9f3467c996b97492232a4b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.607810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:47.621686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cebc083b9f3467c996b97492232a4b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.643727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cebc083b9f3467c996b97492232a4b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.644692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cebc083b9f3467c996b97492232a4b4'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:47.655534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb4e00a-ee69-4aad-8cf4-a357c59b7366 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.659424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 945/1868] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:47 2022] DELETE /identity/v3/users/3cebc083b9f3467c996b97492232a4b4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:47.660321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.661728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.717819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:47.751610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.753027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.753602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.754052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.757495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.761777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381bcf0-728c-4d33-a531-f9167b5a00bb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.767828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b5b359-bcde-48ab-be09-f8b8fd0b70a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.769321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5b359-bcde-48ab-be09-f8b8fd0b70a0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.769321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5b359-bcde-48ab-be09-f8b8fd0b70a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.769509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b5b359-bcde-48ab-be09-f8b8fd0b70a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.770296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5b359-bcde-48ab-be09-f8b8fd0b70a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:47.771061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5b359-bcde-48ab-be09-f8b8fd0b70a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:47.773030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b5b359-bcde-48ab-be09-f8b8fd0b70a0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:47.792527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 946/1869] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:47.792527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d92580-7165-46e6-af0d-76143f114ebf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.792527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d92580-7165-46e6-af0d-76143f114ebf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.792527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d92580-7165-46e6-af0d-76143f114ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.813721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-29d92580-7165-46e6-af0d-76143f114ebf None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:47.814985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-29d92580-7165-46e6-af0d-76143f114ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.823051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 947/1870] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:47.826423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d92580-7165-46e6-af0d-76143f114ebf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.837062 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d92580-7165-46e6-af0d-76143f114ebf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.842142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaaf33-f5bb-432c-8a03-6f02921ee659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.843177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaaf33-f5bb-432c-8a03-6f02921ee659 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.843553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaaf33-f5bb-432c-8a03-6f02921ee659 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.844163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6faaaf33-f5bb-432c-8a03-6f02921ee659 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.846062 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaaf33-f5bb-432c-8a03-6f02921ee659 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:47.849719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6faaaf33-f5bb-432c-8a03-6f02921ee659 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:47.854815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:47.856308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:47.866827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 924/1871] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:47 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:47.875464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.877446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.891343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c44a25-dab7-479d-87c7-0ed94c3de641 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:47.893080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 948/1872] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:47 2022] POST /identity/v3/projects => generated 371 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:47.894294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaaf33-f5bb-432c-8a03-6f02921ee659 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:47.897273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6faaaf33-f5bb-432c-8a03-6f02921ee659 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.961071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22405bc0-9aa0-420a-a095-7b855036f274 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:47.961960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22405bc0-9aa0-420a-a095-7b855036f274 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:47.962636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22405bc0-9aa0-420a-a095-7b855036f274 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:47.963106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22405bc0-9aa0-420a-a095-7b855036f274 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:47.965663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-22405bc0-9aa0-420a-a095-7b855036f274 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:47.996131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6faaaf33-f5bb-432c-8a03-6f02921ee659 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:48.002271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab193c1c-3933-4f92-9135-e3c435eb3bcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:48.004954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab193c1c-3933-4f92-9135-e3c435eb3bcc service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:48.005651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab193c1c-3933-4f92-9135-e3c435eb3bcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:48.006071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab193c1c-3933-4f92-9135-e3c435eb3bcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:48.131716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22405bc0-9aa0-420a-a095-7b855036f274 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:48.137274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22405bc0-9aa0-420a-a095-7b855036f274 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:48.140460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:48.164008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 925/1873] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:47 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:48.171208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22405bc0-9aa0-420a-a095-7b855036f274 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:48.186547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab193c1c-3933-4f92-9135-e3c435eb3bcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:48.188139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab193c1c-3933-4f92-9135-e3c435eb3bcc service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:48.213655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22405bc0-9aa0-420a-a095-7b855036f274 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:48.220957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98293bef-0ddd-461f-bf9a-80073d49ec44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:48.221883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98293bef-0ddd-461f-bf9a-80073d49ec44 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:48.221883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98293bef-0ddd-461f-bf9a-80073d49ec44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:48.222124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98293bef-0ddd-461f-bf9a-80073d49ec44 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:48.222788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98293bef-0ddd-461f-bf9a-80073d49ec44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:48.222920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 949/1874] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:48.225242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98293bef-0ddd-461f-bf9a-80073d49ec44 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:48.318863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 926/1875] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:48 2022] POST /identity/v3/users => generated 286 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:48.321531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab193c1c-3933-4f92-9135-e3c435eb3bcc service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:48.393388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab193c1c-3933-4f92-9135-e3c435eb3bcc service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:48.428293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146e19d-09d8-4837-8b98-164811cef9e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:48.429057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146e19d-09d8-4837-8b98-164811cef9e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:48.429400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146e19d-09d8-4837-8b98-164811cef9e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:48.429764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146e19d-09d8-4837-8b98-164811cef9e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:48.430562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146e19d-09d8-4837-8b98-164811cef9e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:48.432909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146e19d-09d8-4837-8b98-164811cef9e0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:48.445044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 950/1876] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:48.450150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98293bef-0ddd-461f-bf9a-80073d49ec44 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:48.468127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98293bef-0ddd-461f-bf9a-80073d49ec44 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:48.491826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52c14dc-d9c4-4b37-be7f-e76d6ef32e07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:48.492487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c14dc-d9c4-4b37-be7f-e76d6ef32e07 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:48.492770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c14dc-d9c4-4b37-be7f-e76d6ef32e07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:48.493045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52c14dc-d9c4-4b37-be7f-e76d6ef32e07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:48.493714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c14dc-d9c4-4b37-be7f-e76d6ef32e07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:48.495755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b52c14dc-d9c4-4b37-be7f-e76d6ef32e07 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:48.533538 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 927/1877] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:48.535398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146e19d-09d8-4837-8b98-164811cef9e0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:48.555338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146e19d-09d8-4837-8b98-164811cef9e0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:48.573723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c54620-50a6-4085-a2be-481e382c8024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:48.575388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c54620-50a6-4085-a2be-481e382c8024 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:48.576367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c54620-50a6-4085-a2be-481e382c8024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:48.576717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c54620-50a6-4085-a2be-481e382c8024 admin admin] PATH_INFO: `/v3/projects/81082b2f3a984d09baac046de385ea96/users/5bd897d855c64ae28ec3f10f69e9e8b2/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:48.647562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c54620-50a6-4085-a2be-481e382c8024 admin admin] RBAC: Authorizing `identity:create_grant(project_id=81082b2f3a984d09baac046de385ea96, user_id=5bd897d855c64ae28ec3f10f69e9e8b2, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:48.659244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c54620-50a6-4085-a2be-481e382c8024 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:48.704037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 951/1878] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:48 2022] PUT /identity/v3/projects/81082b2f3a984d09baac046de385ea96/users/5bd897d855c64ae28ec3f10f69e9e8b2/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:48.709821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb162a80-f57b-4180-be3e-92add0e7b93b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:48.710558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb162a80-f57b-4180-be3e-92add0e7b93b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:48.711008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb162a80-f57b-4180-be3e-92add0e7b93b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:48.763436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cb162a80-f57b-4180-be3e-92add0e7b93b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:48.764692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cb162a80-f57b-4180-be3e-92add0e7b93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:48.780687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:48.926259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 928/1879] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:48.937227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c54620-50a6-4085-a2be-481e382c8024 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:48.940716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-33c54620-50a6-4085-a2be-481e382c8024 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:49.041924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb162a80-f57b-4180-be3e-92add0e7b93b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:49.051254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cb162a80-f57b-4180-be3e-92add0e7b93b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:49.083443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb162a80-f57b-4180-be3e-92add0e7b93b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:49.096463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:49.101084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:49.101457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:49.102011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] PATH_INFO: `/v3/projects/c6cf225f61ed4f0bbcdb9533d608a947` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:49.103441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6cf225f61ed4f0bbcdb9533d608a947)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:49.107523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:49.164717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c54620-50a6-4085-a2be-481e382c8024 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:49.181348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712a541-3d93-45f2-b9c6-502c058f1450 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:49.182338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712a541-3d93-45f2-b9c6-502c058f1450 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:49.182613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712a541-3d93-45f2-b9c6-502c058f1450 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:49.183029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a712a541-3d93-45f2-b9c6-502c058f1450 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:49.190369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6cf225f61ed4f0bbcdb9533d608a947', 'request_id': 'req-8ca1c942-3e11-46b7-a83d-43975c0c3250'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.199171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6cf225f61ed4f0bbcdb9533d608a947', 'request_id': 'req-8ca1c942-3e11-46b7-a83d-43975c0c3250'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.213144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6cf225f61ed4f0bbcdb9533d608a947', 'request_id': 'req-8ca1c942-3e11-46b7-a83d-43975c0c3250'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.213789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712a541-3d93-45f2-b9c6-502c058f1450 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:49.213789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a712a541-3d93-45f2-b9c6-502c058f1450 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:49.225061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 952/1880] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:49.233167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] The token cache is being invalidate because project c6cf225f61ed4f0bbcdb9533d608a947 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:49.233508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.234280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.242996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 929/1881] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:49 2022] DELETE /identity/v3/projects/c6cf225f61ed4f0bbcdb9533d608a947 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:49.247179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712a541-3d93-45f2-b9c6-502c058f1450 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:49.259178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a712a541-3d93-45f2-b9c6-502c058f1450 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:49.350006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:49.350781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:49.351069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:49.351818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] PATH_INFO: `/v3/users/17d56e5731664f82b6e53693eec68c3c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:49.353837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17d56e5731664f82b6e53693eec68c3c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:49.357708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:49.478775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17d56e5731664f82b6e53693eec68c3c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.479802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17d56e5731664f82b6e53693eec68c3c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.494676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17d56e5731664f82b6e53693eec68c3c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.496045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:49.530885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17d56e5731664f82b6e53693eec68c3c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.531965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17d56e5731664f82b6e53693eec68c3c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:49.550045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 953/1882] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:49 2022] DELETE /identity/v3/users/17d56e5731664f82b6e53693eec68c3c => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:50.245972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:50.248479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:50.279200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca1c942-3e11-46b7-a83d-43975c0c3250 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:50.416768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:50.417437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:50.417842 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:50.418044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] PATH_INFO: `/v3/projects/56bdc0a62ae0474e80f5d02fe8d70b40` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:50.419239 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56bdc0a62ae0474e80f5d02fe8d70b40)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:50.421309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:50.463436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:50.479476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56bdc0a62ae0474e80f5d02fe8d70b40', 'request_id': 'req-36446972-1fc4-4f63-b5e6-0e0b83af33a6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.487606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56bdc0a62ae0474e80f5d02fe8d70b40', 'request_id': 'req-36446972-1fc4-4f63-b5e6-0e0b83af33a6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.493768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56bdc0a62ae0474e80f5d02fe8d70b40', 'request_id': 'req-36446972-1fc4-4f63-b5e6-0e0b83af33a6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.499885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:50.501309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:50.508465 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] The token cache is being invalidate because project 56bdc0a62ae0474e80f5d02fe8d70b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:50.508892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.509736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36446972-1fc4-4f63-b5e6-0e0b83af33a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.511740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 930/1883] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:50 2022] DELETE /identity/v3/projects/56bdc0a62ae0474e80f5d02fe8d70b40 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:50.516339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3410-457e-465b-8633-82264b4f540e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:50.516733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3410-457e-465b-8633-82264b4f540e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:50.517074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02a3410-457e-465b-8633-82264b4f540e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:50.519476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1b66e-6c98-40d8-9bda-dc3865e7c112 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:50.540615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d02a3410-457e-465b-8633-82264b4f540e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:50.542003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d02a3410-457e-465b-8633-82264b4f540e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:50.600805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:50.601991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:50.602714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:50.602714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] PATH_INFO: `/v3/users/3391943c06c6409eb8262038b78f5949` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:50.603862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3391943c06c6409eb8262038b78f5949)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:50.607522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:50.633725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 931/1884] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:50.642338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb60fa5-0b93-4791-a3c0-ab32cf6f93fb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:50.642711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb60fa5-0b93-4791-a3c0-ab32cf6f93fb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:50.643181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb60fa5-0b93-4791-a3c0-ab32cf6f93fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:50.676979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3391943c06c6409eb8262038b78f5949'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.677752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3391943c06c6409eb8262038b78f5949'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.678970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9fb60fa5-0b93-4791-a3c0-ab32cf6f93fb None None] MFA Rules not processed for user `5bd897d855c64ae28ec3f10f69e9e8b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:50.680709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9fb60fa5-0b93-4791-a3c0-ab32cf6f93fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:50.689725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3391943c06c6409eb8262038b78f5949'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.706458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3391943c06c6409eb8262038b78f5949'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.710480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b7fb16f6-f0fd-45a3-9b43-54bc539e5d15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3391943c06c6409eb8262038b78f5949'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:50.731175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 954/1885] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:50 2022] DELETE /identity/v3/users/3391943c06c6409eb8262038b78f5949 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:50.733398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c828697-18f7-4206-a816-e10dc8ace103 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:50.734206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c828697-18f7-4206-a816-e10dc8ace103 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:50.734624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c828697-18f7-4206-a816-e10dc8ace103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:50.764595 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3c828697-18f7-4206-a816-e10dc8ace103 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:50.771511 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 932/1886] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:50 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:50.774871 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e996a4-0a3d-493f-9975-e1367304cda9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:50.775574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e996a4-0a3d-493f-9975-e1367304cda9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:50.776672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e996a4-0a3d-493f-9975-e1367304cda9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:50.777268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3c828697-18f7-4206-a816-e10dc8ace103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:50.816853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-41e996a4-0a3d-493f-9975-e1367304cda9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:50.820427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-41e996a4-0a3d-493f-9975-e1367304cda9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:50.887300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:50.913051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 955/1887] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:50.914149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c828697-18f7-4206-a816-e10dc8ace103 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:50.915542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3c828697-18f7-4206-a816-e10dc8ace103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:50.935679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:50.955513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 933/1888] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:50.955513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e996a4-0a3d-493f-9975-e1367304cda9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:50.985191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e996a4-0a3d-493f-9975-e1367304cda9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:50.997146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:50.999255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:50.999649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.000184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.002172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:51.002564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.004445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.019798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c828697-18f7-4206-a816-e10dc8ace103 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.029385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 934/1889] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:51.030432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:51.032077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db770858-9a39-4278-a504-fa66546e011c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.032935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db770858-9a39-4278-a504-fa66546e011c service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.033581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db770858-9a39-4278-a504-fa66546e011c service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.034149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db770858-9a39-4278-a504-fa66546e011c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.034516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:51.120282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db770858-9a39-4278-a504-fa66546e011c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.124471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db770858-9a39-4278-a504-fa66546e011c service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.139229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 956/1890] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:50 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:51.140207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db770858-9a39-4278-a504-fa66546e011c service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:51.158814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db770858-9a39-4278-a504-fa66546e011c service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.172670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c1218-b6ce-47b7-ad3c-dc1f58c40fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.173556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c1218-b6ce-47b7-ad3c-dc1f58c40fcb admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.174013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c1218-b6ce-47b7-ad3c-dc1f58c40fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.174508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356c1218-b6ce-47b7-ad3c-dc1f58c40fcb admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.175283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c1218-b6ce-47b7-ad3c-dc1f58c40fcb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.181639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356c1218-b6ce-47b7-ad3c-dc1f58c40fcb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.204542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd757b5-92c0-496a-aa98-1df383941d4a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.220629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10bc76-3932-49f3-b772-4c3eccf78d11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.221283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10bc76-3932-49f3-b772-4c3eccf78d11 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.221554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10bc76-3932-49f3-b772-4c3eccf78d11 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.221817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca10bc76-3932-49f3-b772-4c3eccf78d11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.233949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10bc76-3932-49f3-b772-4c3eccf78d11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.235382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca10bc76-3932-49f3-b772-4c3eccf78d11 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.238893 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 957/1891] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:51 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:51.243209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c1218-b6ce-47b7-ad3c-dc1f58c40fcb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:51.251909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 935/1892] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:51.336125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356c1218-b6ce-47b7-ad3c-dc1f58c40fcb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.342695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3240b40-e92c-4c4d-ad9d-ffca2b4f0c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.343331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3240b40-e92c-4c4d-ad9d-ffca2b4f0c54 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.343758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3240b40-e92c-4c4d-ad9d-ffca2b4f0c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.344011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3240b40-e92c-4c4d-ad9d-ffca2b4f0c54 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.344978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3240b40-e92c-4c4d-ad9d-ffca2b4f0c54 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.347530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3240b40-e92c-4c4d-ad9d-ffca2b4f0c54 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.388059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10bc76-3932-49f3-b772-4c3eccf78d11 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:51.397790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 958/1893] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:51 2022] POST /identity/v3/users => generated 287 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:01:51.402564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca10bc76-3932-49f3-b772-4c3eccf78d11 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.409504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a0762-543b-4997-bf15-0942a60abecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.410276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a0762-543b-4997-bf15-0942a60abecd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.410702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a0762-543b-4997-bf15-0942a60abecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.411098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537a0762-543b-4997-bf15-0942a60abecd admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.412178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a0762-543b-4997-bf15-0942a60abecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.413850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537a0762-543b-4997-bf15-0942a60abecd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.426286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 936/1894] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:51.435938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3240b40-e92c-4c4d-ad9d-ffca2b4f0c54 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:51.451194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3240b40-e92c-4c4d-ad9d-ffca2b4f0c54 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.459140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4179b3-df36-417d-af3f-dfef41821a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.459892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4179b3-df36-417d-af3f-dfef41821a8e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.460898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4179b3-df36-417d-af3f-dfef41821a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.461284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4179b3-df36-417d-af3f-dfef41821a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.462632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4179b3-df36-417d-af3f-dfef41821a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.465180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4179b3-df36-417d-af3f-dfef41821a8e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.483932 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 959/1895] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:51 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:51.484570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a0762-543b-4997-bf15-0942a60abecd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:51.485089 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4179b3-df36-417d-af3f-dfef41821a8e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:51.487188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0f4179b3-df36-417d-af3f-dfef41821a8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:51.496296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:51.504232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537a0762-543b-4997-bf15-0942a60abecd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.504685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4179b3-df36-417d-af3f-dfef41821a8e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.512123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.513572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.514324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.514807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] PATH_INFO: `/v3/users/6da40f665b304c0a8c3512f024d1481d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.514942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7eca61-daa2-4a6c-b685-97f6aa160283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.515596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eca61-daa2-4a6c-b685-97f6aa160283 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.515882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eca61-daa2-4a6c-b685-97f6aa160283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.516246 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7eca61-daa2-4a6c-b685-97f6aa160283 admin admin] PATH_INFO: `/v3/projects/a47f29ce6a804447902c8586d7ff7ef4/users/6a9f76f12d8846eb96c40ccce220b3f4/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.517197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6da40f665b304c0a8c3512f024d1481d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.520235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.537480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eca61-daa2-4a6c-b685-97f6aa160283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a47f29ce6a804447902c8586d7ff7ef4, user_id=6a9f76f12d8846eb96c40ccce220b3f4, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.541097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7eca61-daa2-4a6c-b685-97f6aa160283 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.572320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 937/1896] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:51 2022] PUT /identity/v3/projects/a47f29ce6a804447902c8586d7ff7ef4/users/6a9f76f12d8846eb96c40ccce220b3f4/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:51.584981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9e9cf-bcd0-4a8a-abaa-21a70d12d5ba None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.585483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9e9cf-bcd0-4a8a-abaa-21a70d12d5ba None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.585769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd9e9cf-bcd0-4a8a-abaa-21a70d12d5ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.595680 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6da40f665b304c0a8c3512f024d1481d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:51.596360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6da40f665b304c0a8c3512f024d1481d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:51.608326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6da40f665b304c0a8c3512f024d1481d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:51.626937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6da40f665b304c0a8c3512f024d1481d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:51.627741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ab1509c3-67f7-49e9-a232-42037525f817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6da40f665b304c0a8c3512f024d1481d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:51.633768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-abd9e9cf-bcd0-4a8a-abaa-21a70d12d5ba None None] MFA Rules not processed for user `6a9f76f12d8846eb96c40ccce220b3f4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:51.635076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-abd9e9cf-bcd0-4a8a-abaa-21a70d12d5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:51.675320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 960/1897] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:51 2022] DELETE /identity/v3/users/6da40f665b304c0a8c3512f024d1481d => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:51.682535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f0170-af3f-41d8-8ad4-bee9330c8c89 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.682968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f0170-af3f-41d8-8ad4-bee9330c8c89 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.683297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4f0170-af3f-41d8-8ad4-bee9330c8c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.716055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2a4f0170-af3f-41d8-8ad4-bee9330c8c89 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:51.717709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2a4f0170-af3f-41d8-8ad4-bee9330c8c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:51.758290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 938/1898] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:51.770719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9e9cf-bcd0-4a8a-abaa-21a70d12d5ba None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:51.772780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-abd9e9cf-bcd0-4a8a-abaa-21a70d12d5ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:51.835290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 961/1899] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:51.860151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:51.865569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f0170-af3f-41d8-8ad4-bee9330c8c89 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:51.868090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2a4f0170-af3f-41d8-8ad4-bee9330c8c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:51.891615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd9e9cf-bcd0-4a8a-abaa-21a70d12d5ba None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.942761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4f0170-af3f-41d8-8ad4-bee9330c8c89 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:51.948062 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c56cc-528b-4c79-b9d1-d63ccbe7ae72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.949127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c56cc-528b-4c79-b9d1-d63ccbe7ae72 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.949240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c56cc-528b-4c79-b9d1-d63ccbe7ae72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.949516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c56cc-528b-4c79-b9d1-d63ccbe7ae72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.960909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c56cc-528b-4c79-b9d1-d63ccbe7ae72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:51.963439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c56cc-528b-4c79-b9d1-d63ccbe7ae72 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:51.968724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da1606-a2bc-4f61-b555-bdea8d5bd62f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:51.970187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da1606-a2bc-4f61-b555-bdea8d5bd62f service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:51.970659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da1606-a2bc-4f61-b555-bdea8d5bd62f service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:51.971328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0da1606-a2bc-4f61-b555-bdea8d5bd62f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:51.975751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 962/1900] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:52.045310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da1606-a2bc-4f61-b555-bdea8d5bd62f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:52.048681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0da1606-a2bc-4f61-b555-bdea8d5bd62f service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:52.051113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c56cc-528b-4c79-b9d1-d63ccbe7ae72 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:52.053504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d42c56cc-528b-4c79-b9d1-d63ccbe7ae72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:52.057864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 939/1901] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:52.078588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c56cc-528b-4c79-b9d1-d63ccbe7ae72 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:52.086890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da1606-a2bc-4f61-b555-bdea8d5bd62f service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:52.088452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b0da1606-a2bc-4f61-b555-bdea8d5bd62f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:52.095853 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:52.097073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:52.097544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:52.098023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] PATH_INFO: `/v3/projects/52fa82c56bf44a8badd6d1697ec5b2bc` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:52.100917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] RBAC: Authorizing `identity:delete_project(project_id=52fa82c56bf44a8badd6d1697ec5b2bc)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:52.104703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:52.112818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0da1606-a2bc-4f61-b555-bdea8d5bd62f service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:52.122939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:52.123864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:52.124375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:52.124898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] PATH_INFO: `/v3/users/6a9f76f12d8846eb96c40ccce220b3f4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:52.126664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a9f76f12d8846eb96c40ccce220b3f4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:52.128754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:52.130787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:52.195111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52fa82c56bf44a8badd6d1697ec5b2bc', 'request_id': 'req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.209502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52fa82c56bf44a8badd6d1697ec5b2bc', 'request_id': 'req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.218373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52fa82c56bf44a8badd6d1697ec5b2bc', 'request_id': 'req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.226551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a9f76f12d8846eb96c40ccce220b3f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.237827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] The token cache is being invalidate because project 52fa82c56bf44a8badd6d1697ec5b2bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:52.238423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.239310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.241043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a9f76f12d8846eb96c40ccce220b3f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.245128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 963/1902] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:52 2022] DELETE /identity/v3/projects/52fa82c56bf44a8badd6d1697ec5b2bc => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:52.245822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:52.247501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:52.263593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a9f76f12d8846eb96c40ccce220b3f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.264454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a9f76f12d8846eb96c40ccce220b3f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.275425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d8101494-042c-4f0c-a562-3db1465803bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a9f76f12d8846eb96c40ccce220b3f4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.285103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 940/1903] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:52 2022] DELETE /identity/v3/users/6a9f76f12d8846eb96c40ccce220b3f4 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:52.289866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd933de6-4994-443d-bbc3-d4b171aacef5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:52.290772 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd933de6-4994-443d-bbc3-d4b171aacef5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:52.291126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd933de6-4994-443d-bbc3-d4b171aacef5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:52.322323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cd933de6-4994-443d-bbc3-d4b171aacef5 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:52.323903 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cd933de6-4994-443d-bbc3-d4b171aacef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:52.331600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b7b2c3-ec48-4b5e-8771-7f3368ab9f90 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:52.375801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:52.425919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:52.427637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:52.428183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:52.428694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] PATH_INFO: `/v3/users/76cf5ed34d5045a6af07bb96b5016d8f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:52.430036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76cf5ed34d5045a6af07bb96b5016d8f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:52.432646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:52.448759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 941/1904] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:52.450038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd933de6-4994-443d-bbc3-d4b171aacef5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:52.454759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cd933de6-4994-443d-bbc3-d4b171aacef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:52.468505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd933de6-4994-443d-bbc3-d4b171aacef5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:52.476888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:52.477574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:52.477854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:52.478153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] PATH_INFO: `/v3/projects/1cea8a7dac1342b8906bfe039916b617` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:52.479414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1cea8a7dac1342b8906bfe039916b617)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:52.481381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:52.493666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76cf5ed34d5045a6af07bb96b5016d8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.494493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76cf5ed34d5045a6af07bb96b5016d8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.498035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:52.510704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76cf5ed34d5045a6af07bb96b5016d8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.526356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76cf5ed34d5045a6af07bb96b5016d8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.527096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76cf5ed34d5045a6af07bb96b5016d8f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.536836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1cea8a7dac1342b8906bfe039916b617', 'request_id': 'req-3325d9d8-01bc-40e4-9b17-25308a675b49'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.541620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 964/1905] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:52 2022] DELETE /identity/v3/users/76cf5ed34d5045a6af07bb96b5016d8f => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:52.542543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:52.543859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:52.545282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1cea8a7dac1342b8906bfe039916b617', 'request_id': 'req-3325d9d8-01bc-40e4-9b17-25308a675b49'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.553206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1cea8a7dac1342b8906bfe039916b617', 'request_id': 'req-3325d9d8-01bc-40e4-9b17-25308a675b49'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.574725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] The token cache is being invalidate because project 1cea8a7dac1342b8906bfe039916b617 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:52.575091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.576166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.578180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 942/1906] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:52 2022] DELETE /identity/v3/projects/1cea8a7dac1342b8906bfe039916b617 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:52.582599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:52.585564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:52.648792 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c8df9-3ed4-4a7d-a427-e57427436f57 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:52.653452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4838eb-d79e-4592-9673-d9ba3af89a8e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:52.655004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4838eb-d79e-4592-9673-d9ba3af89a8e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:52.655442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4838eb-d79e-4592-9673-d9ba3af89a8e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:52.656455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4838eb-d79e-4592-9673-d9ba3af89a8e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:52.658777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ba4838eb-d79e-4592-9673-d9ba3af89a8e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:52.675475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3325d9d8-01bc-40e4-9b17-25308a675b49 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:52.743861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4838eb-d79e-4592-9673-d9ba3af89a8e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:52.749221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4838eb-d79e-4592-9673-d9ba3af89a8e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:52.767494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 965/1907] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:52.777283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:52.789761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:52.791512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:52.793037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:52.793534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] PATH_INFO: `/v3/users/948d2845cb00403e83e1dc12db9ea8ec` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:52.795055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=948d2845cb00403e83e1dc12db9ea8ec)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:52.797414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:52.834126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87947d6d-ac42-4747-a46e-2e9d2aa84fa8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:52.834436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87947d6d-ac42-4747-a46e-2e9d2aa84fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:52.834709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87947d6d-ac42-4747-a46e-2e9d2aa84fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:52.861470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-87947d6d-ac42-4747-a46e-2e9d2aa84fa8 None None] MFA Rules not processed for user `5bd897d855c64ae28ec3f10f69e9e8b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:52.863169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-87947d6d-ac42-4747-a46e-2e9d2aa84fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:52.894191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '948d2845cb00403e83e1dc12db9ea8ec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.907037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '948d2845cb00403e83e1dc12db9ea8ec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.929156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '948d2845cb00403e83e1dc12db9ea8ec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.929973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '948d2845cb00403e83e1dc12db9ea8ec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.941067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '948d2845cb00403e83e1dc12db9ea8ec'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:52.949960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 943/1908] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:52 2022] DELETE /identity/v3/users/948d2845cb00403e83e1dc12db9ea8ec => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:52.958378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 966/1909] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:52 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:53.382601 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:53.384708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:53.554488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03357b1c-340f-40b7-93a6-89cda6e83ca7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:53.560304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:53.560960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:53.561242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:53.561942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] PATH_INFO: `/v3/projects/c30ac5fa774948be87174d2ce4e05bdf` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:53.563021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30ac5fa774948be87174d2ce4e05bdf)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:53.564955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:53.632139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87947d6d-ac42-4747-a46e-2e9d2aa84fa8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:53.634304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-87947d6d-ac42-4747-a46e-2e9d2aa84fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:53.646310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30ac5fa774948be87174d2ce4e05bdf', 'request_id': 'req-2438dfe3-4938-44ff-922d-1abedb5b8983'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.655653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87947d6d-ac42-4747-a46e-2e9d2aa84fa8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:53.672182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30ac5fa774948be87174d2ce4e05bdf', 'request_id': 'req-2438dfe3-4938-44ff-922d-1abedb5b8983'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.686357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=3,keystone:SELECT=97 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:53.689245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30ac5fa774948be87174d2ce4e05bdf', 'request_id': 'req-2438dfe3-4938-44ff-922d-1abedb5b8983'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.709533 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] The token cache is being invalidate because project c30ac5fa774948be87174d2ce4e05bdf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:53.709838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.710546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.712452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 944/1910] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:53 2022] DELETE /identity/v3/projects/c30ac5fa774948be87174d2ce4e05bdf => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:53.713659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:53.728492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:53.735044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:53.757766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:53.758556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:53.758839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:53.759117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] PATH_INFO: `/v3/projects/a47f29ce6a804447902c8586d7ff7ef4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:53.760522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a47f29ce6a804447902c8586d7ff7ef4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:53.762816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:53.825697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2438dfe3-4938-44ff-922d-1abedb5b8983 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:53.826438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a47f29ce6a804447902c8586d7ff7ef4', 'request_id': 'req-03bb2279-2109-4f68-9c88-5f6d7055fca6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.834610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a47f29ce6a804447902c8586d7ff7ef4', 'request_id': 'req-03bb2279-2109-4f68-9c88-5f6d7055fca6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.842274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a47f29ce6a804447902c8586d7ff7ef4', 'request_id': 'req-03bb2279-2109-4f68-9c88-5f6d7055fca6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.861124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] The token cache is being invalidate because project a47f29ce6a804447902c8586d7ff7ef4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:53.861419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.862106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-03bb2279-2109-4f68-9c88-5f6d7055fca6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.864091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 967/1911] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:53 2022] DELETE /identity/v3/projects/a47f29ce6a804447902c8586d7ff7ef4 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:53.865460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288c1ee-0137-4735-94ab-38179e85376c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:53.865915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288c1ee-0137-4735-94ab-38179e85376c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:53.866212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8288c1ee-0137-4735-94ab-38179e85376c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:53.887036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8288c1ee-0137-4735-94ab-38179e85376c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:53.888889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8288c1ee-0137-4735-94ab-38179e85376c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:53.903248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:53.904106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:53.904454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:53.904791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] PATH_INFO: `/v3/projects/f9164cf694de45679c0e8d88badf9730` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:53.906000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9164cf694de45679c0e8d88badf9730)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:53.908100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:53.957946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 968/1912] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:53.959712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee10a1-0c66-48fe-85bb-7d850709f4b6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:53.959998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee10a1-0c66-48fe-85bb-7d850709f4b6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:53.960272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee10a1-0c66-48fe-85bb-7d850709f4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:53.971334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9164cf694de45679c0e8d88badf9730', 'request_id': 'req-c2eae325-a13a-49f1-97d7-2064f6bfc17f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.980205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9164cf694de45679c0e8d88badf9730', 'request_id': 'req-c2eae325-a13a-49f1-97d7-2064f6bfc17f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:53.983169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-30ee10a1-0c66-48fe-85bb-7d850709f4b6 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:53.984769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-30ee10a1-0c66-48fe-85bb-7d850709f4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:53.990755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9164cf694de45679c0e8d88badf9730', 'request_id': 'req-c2eae325-a13a-49f1-97d7-2064f6bfc17f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:54.010747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] The token cache is being invalidate because project f9164cf694de45679c0e8d88badf9730 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:54.011278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:54.012135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c2eae325-a13a-49f1-97d7-2064f6bfc17f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:54.014237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 945/1913] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:53 2022] DELETE /identity/v3/projects/f9164cf694de45679c0e8d88badf9730 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:54.015757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a42e30-0e2c-4890-8f0d-68ad59686ee5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.016182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a42e30-0e2c-4890-8f0d-68ad59686ee5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.016708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a42e30-0e2c-4890-8f0d-68ad59686ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.041268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d5a42e30-0e2c-4890-8f0d-68ad59686ee5 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:54.041819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:54.042748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d5a42e30-0e2c-4890-8f0d-68ad59686ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:54.066231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 969/1914] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:54.067837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee10a1-0c66-48fe-85bb-7d850709f4b6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.069332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-30ee10a1-0c66-48fe-85bb-7d850709f4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:54.080885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:54.093361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee10a1-0c66-48fe-85bb-7d850709f4b6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.100010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.100825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.101896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.102432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] PATH_INFO: `/v3/users/bc2cc0b39fdf476fa45fb845cf50cd8b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.103796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc2cc0b39fdf476fa45fb845cf50cd8b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.107082 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.117449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 946/1915] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:54.119327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae22ac3-a734-485a-a812-5080bafc8d9f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.119662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae22ac3-a734-485a-a812-5080bafc8d9f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.119941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae22ac3-a734-485a-a812-5080bafc8d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.158297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fae22ac3-a734-485a-a812-5080bafc8d9f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:54.159544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fae22ac3-a734-485a-a812-5080bafc8d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:54.172799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 947/1916] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:54.174148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae22ac3-a734-485a-a812-5080bafc8d9f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.187096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae22ac3-a734-485a-a812-5080bafc8d9f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.193509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c64c0-a9f3-47c6-95a3-5af848b5337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.194638 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c64c0-a9f3-47c6-95a3-5af848b5337e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.195032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c64c0-a9f3-47c6-95a3-5af848b5337e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.195569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284c64c0-a9f3-47c6-95a3-5af848b5337e admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.196508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c64c0-a9f3-47c6-95a3-5af848b5337e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:54.197023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c64c0-a9f3-47c6-95a3-5af848b5337e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.198043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc2cc0b39fdf476fa45fb845cf50cd8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:54.198978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc2cc0b39fdf476fa45fb845cf50cd8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:54.199105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284c64c0-a9f3-47c6-95a3-5af848b5337e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.210534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc2cc0b39fdf476fa45fb845cf50cd8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:54.215161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 948/1917] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:54.219594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c64c0-a9f3-47c6-95a3-5af848b5337e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.227001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc2cc0b39fdf476fa45fb845cf50cd8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:54.227821 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc2cc0b39fdf476fa45fb845cf50cd8b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:54.233796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284c64c0-a9f3-47c6-95a3-5af848b5337e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.239670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.240381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.240660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.240989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.241573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:54.241872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.244548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.244663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 970/1918] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:54 2022] DELETE /identity/v3/users/bc2cc0b39fdf476fa45fb845cf50cd8b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:54.245922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.248361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:54.281032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 949/1919] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:54.286721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.290914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:54.321375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d174f-66b6-4dd7-9318-2ca56c3dec34 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.343507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307dcd0-2af4-4b2b-bdda-5a4522c1a278 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.352969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cf205-d0ea-4134-a0a1-89c5a63001b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.353920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cf205-d0ea-4134-a0a1-89c5a63001b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.354437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cf205-d0ea-4134-a0a1-89c5a63001b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.354926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8cf205-d0ea-4134-a0a1-89c5a63001b8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.356560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cf205-d0ea-4134-a0a1-89c5a63001b8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.360976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8cf205-d0ea-4134-a0a1-89c5a63001b8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.412184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 971/1920] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:54 2022] POST /identity/v3/projects => generated 381 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:54.416103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cf205-d0ea-4134-a0a1-89c5a63001b8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.456677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5139115-67b5-4002-80e9-acf010b5920b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.457495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5139115-67b5-4002-80e9-acf010b5920b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.457997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5139115-67b5-4002-80e9-acf010b5920b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.460547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5139115-67b5-4002-80e9-acf010b5920b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.463433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5139115-67b5-4002-80e9-acf010b5920b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.468002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5139115-67b5-4002-80e9-acf010b5920b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.487284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:54.502673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8cf205-d0ea-4134-a0a1-89c5a63001b8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.508988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc93711-0b77-41ac-ac80-d3ec307d7963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.509991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc93711-0b77-41ac-ac80-d3ec307d7963 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.510657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc93711-0b77-41ac-ac80-d3ec307d7963 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.511069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc93711-0b77-41ac-ac80-d3ec307d7963 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.512044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc93711-0b77-41ac-ac80-d3ec307d7963 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.514557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc93711-0b77-41ac-ac80-d3ec307d7963 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.542690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 950/1921] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:54 2022] POST /identity/v3/projects => generated 357 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:54.549054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5139115-67b5-4002-80e9-acf010b5920b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.564200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 972/1922] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:54 2022] POST /identity/v3/users => generated 291 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:54.568500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc93711-0b77-41ac-ac80-d3ec307d7963 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.588309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc93711-0b77-41ac-ac80-d3ec307d7963 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.705753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5139115-67b5-4002-80e9-acf010b5920b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.712205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f5e17-ef93-4e90-ae89-9dd18a780ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.730627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f5e17-ef93-4e90-ae89-9dd18a780ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.731089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f5e17-ef93-4e90-ae89-9dd18a780ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.731360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f5e17-ef93-4e90-ae89-9dd18a780ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.732687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f5e17-ef93-4e90-ae89-9dd18a780ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.735030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f5e17-ef93-4e90-ae89-9dd18a780ce9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.774992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6f6f5-2113-4894-82c3-2c69fa878cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.775697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6f6f5-2113-4894-82c3-2c69fa878cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.776054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6f6f5-2113-4894-82c3-2c69fa878cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.776195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb6f6f5-2113-4894-82c3-2c69fa878cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.776879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6f6f5-2113-4894-82c3-2c69fa878cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.778957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb6f6f5-2113-4894-82c3-2c69fa878cc2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.788279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 951/1923] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:54 2022] POST /identity/v3/users => generated 279 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:54.789402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f5e17-ef93-4e90-ae89-9dd18a780ce9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.803984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 973/1924] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:54.808009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6f6f5-2113-4894-82c3-2c69fa878cc2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.812008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f5e17-ef93-4e90-ae89-9dd18a780ce9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.819983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4ca92-d784-491c-928d-21e9e7a000f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.821048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4ca92-d784-491c-928d-21e9e7a000f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.821578 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4ca92-d784-491c-928d-21e9e7a000f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.822211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc4ca92-d784-491c-928d-21e9e7a000f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.823063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4ca92-d784-491c-928d-21e9e7a000f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.825964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc4ca92-d784-491c-928d-21e9e7a000f9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.830417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb6f6f5-2113-4894-82c3-2c69fa878cc2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.836158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a1350-af59-4153-8ed0-0c6bea4e5a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.836838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a1350-af59-4153-8ed0-0c6bea4e5a4c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.837053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a1350-af59-4153-8ed0-0c6bea4e5a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.837334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a1350-af59-4153-8ed0-0c6bea4e5a4c admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.837985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a1350-af59-4153-8ed0-0c6bea4e5a4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.845181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a1350-af59-4153-8ed0-0c6bea4e5a4c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.858724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 952/1925] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:54.859816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4ca92-d784-491c-928d-21e9e7a000f9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.881110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc4ca92-d784-491c-928d-21e9e7a000f9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.882506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 974/1926] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:01:54.883224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a1350-af59-4153-8ed0-0c6bea4e5a4c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.889308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8efab-6b7c-4b32-bb17-9314f4a25a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.890196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8efab-6b7c-4b32-bb17-9314f4a25a2c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.890550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8efab-6b7c-4b32-bb17-9314f4a25a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.891025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c8efab-6b7c-4b32-bb17-9314f4a25a2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.891905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8efab-6b7c-4b32-bb17-9314f4a25a2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.893681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c8efab-6b7c-4b32-bb17-9314f4a25a2c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.902064 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:54.907861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a1350-af59-4153-8ed0-0c6bea4e5a4c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.916969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 953/1927] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:54 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:54.921143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46925384-ed8c-4ddb-9600-781c5d4ee75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.923169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8efab-6b7c-4b32-bb17-9314f4a25a2c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:54.924828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46925384-ed8c-4ddb-9600-781c5d4ee75b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.925102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46925384-ed8c-4ddb-9600-781c5d4ee75b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.926715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46925384-ed8c-4ddb-9600-781c5d4ee75b admin admin] PATH_INFO: `/v3/projects/29798d0dec39434abd773f967881f7a5/users/c704810fc4c14d04a70c5cce5a30cce5/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.945497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c8efab-6b7c-4b32-bb17-9314f4a25a2c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:54.952747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-600b2f2a-9cd9-44da-8f20-e7e6ad728376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:54.952925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46925384-ed8c-4ddb-9600-781c5d4ee75b admin admin] RBAC: Authorizing `identity:create_grant(project_id=29798d0dec39434abd773f967881f7a5, user_id=c704810fc4c14d04a70c5cce5a30cce5, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.953950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b2f2a-9cd9-44da-8f20-e7e6ad728376 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.954337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b2f2a-9cd9-44da-8f20-e7e6ad728376 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.954678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-600b2f2a-9cd9-44da-8f20-e7e6ad728376 admin admin] PATH_INFO: `/v3/projects/8efaeab9973245179f4d3ebc6e8aaf1b/users/9bd5b7c3f997479da0a6dd3def18cb1d/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:54.955702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46925384-ed8c-4ddb-9600-781c5d4ee75b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.975644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b2f2a-9cd9-44da-8f20-e7e6ad728376 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8efaeab9973245179f4d3ebc6e8aaf1b, user_id=9bd5b7c3f997479da0a6dd3def18cb1d, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:54.978898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-600b2f2a-9cd9-44da-8f20-e7e6ad728376 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:54.992569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 975/1928] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:54 2022] PUT /identity/v3/projects/29798d0dec39434abd773f967881f7a5/users/c704810fc4c14d04a70c5cce5a30cce5/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:54.995647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757aa4-abf7-48c7-a1d4-08490b8d2065 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:54.998585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757aa4-abf7-48c7-a1d4-08490b8d2065 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:54.998905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85757aa4-abf7-48c7-a1d4-08490b8d2065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:55.026161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 954/1929] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:54 2022] PUT /identity/v3/projects/8efaeab9973245179f4d3ebc6e8aaf1b/users/9bd5b7c3f997479da0a6dd3def18cb1d/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:55.030454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53354d-add8-4351-87dc-c6046d1ffeca None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:55.030795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53354d-add8-4351-87dc-c6046d1ffeca None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:55.031247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53354d-add8-4351-87dc-c6046d1ffeca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:55.078396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-85757aa4-abf7-48c7-a1d4-08490b8d2065 None None] MFA Rules not processed for user `c704810fc4c14d04a70c5cce5a30cce5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:55.081919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1b53354d-add8-4351-87dc-c6046d1ffeca None None] MFA Rules not processed for user `9bd5b7c3f997479da0a6dd3def18cb1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:55.083561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1b53354d-add8-4351-87dc-c6046d1ffeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:55.083904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85757aa4-abf7-48c7-a1d4-08490b8d2065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:55.223579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 955/1930] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:55 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:55.226046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53354d-add8-4351-87dc-c6046d1ffeca None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:55.234442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1b53354d-add8-4351-87dc-c6046d1ffeca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:55.323969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 976/1931] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:55.325061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757aa4-abf7-48c7-a1d4-08490b8d2065 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:55.335277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-85757aa4-abf7-48c7-a1d4-08490b8d2065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:55.372198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53354d-add8-4351-87dc-c6046d1ffeca None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:55.381850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:55.382369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:55.382900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:55.383346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] PATH_INFO: `/v3/projects/beabf0655a17461d8f09e76132bdfc36` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:55.385300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=beabf0655a17461d8f09e76132bdfc36)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:55.388355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:55.420715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:55.458370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85757aa4-abf7-48c7-a1d4-08490b8d2065 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:55.465584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beabf0655a17461d8f09e76132bdfc36', 'request_id': 'req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:55.474880 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beabf0655a17461d8f09e76132bdfc36', 'request_id': 'req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:55.483783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beabf0655a17461d8f09e76132bdfc36', 'request_id': 'req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:55.488961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:55.524632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] The token cache is being invalidate because project beabf0655a17461d8f09e76132bdfc36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:55.524942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:55.527307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:55.535422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 956/1932] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:55 2022] DELETE /identity/v3/projects/beabf0655a17461d8f09e76132bdfc36 => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:55.536529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:55.538056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:55.541937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b04664a-e80b-4615-b033-40f76b23d399 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:55.543069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04664a-e80b-4615-b033-40f76b23d399 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:55.543433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04664a-e80b-4615-b033-40f76b23d399 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:55.543885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b04664a-e80b-4615-b033-40f76b23d399 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:55.546521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1b04664a-e80b-4615-b033-40f76b23d399 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:55.671137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ca798b-957e-4c3d-ab4f-22ebfc0044ff admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:55.679240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843d208-4adc-449c-8067-54e990c8c266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:55.680358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843d208-4adc-449c-8067-54e990c8c266 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:55.681122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843d208-4adc-449c-8067-54e990c8c266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:55.681542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9843d208-4adc-449c-8067-54e990c8c266 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:55.683168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843d208-4adc-449c-8067-54e990c8c266 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:55.685893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9843d208-4adc-449c-8067-54e990c8c266 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:55.687179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04664a-e80b-4615-b033-40f76b23d399 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:55.688622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b04664a-e80b-4615-b033-40f76b23d399 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:55.712854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 977/1933] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:55 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:55.719182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec96ec3-efd6-425d-8364-dfe6ff0e3ee0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:55.720486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec96ec3-efd6-425d-8364-dfe6ff0e3ee0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:55.720776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec96ec3-efd6-425d-8364-dfe6ff0e3ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:55.734981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 957/1934] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:55 2022] POST /identity/v3/projects => generated 383 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:55.735827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843d208-4adc-449c-8067-54e990c8c266 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:55.741052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9843d208-4adc-449c-8067-54e990c8c266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:55.766007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7ec96ec3-efd6-425d-8364-dfe6ff0e3ee0 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:55.767674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7ec96ec3-efd6-425d-8364-dfe6ff0e3ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:55.845819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 978/1935] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:55.845819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec96ec3-efd6-425d-8364-dfe6ff0e3ee0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:55.868163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec96ec3-efd6-425d-8364-dfe6ff0e3ee0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:55.892200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61d4c50-7e96-4dc3-8d3f-97a2f527cab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:55.894756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d4c50-7e96-4dc3-8d3f-97a2f527cab6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:55.895817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d4c50-7e96-4dc3-8d3f-97a2f527cab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:55.896209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61d4c50-7e96-4dc3-8d3f-97a2f527cab6 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:55.898309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d4c50-7e96-4dc3-8d3f-97a2f527cab6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:55.902539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61d4c50-7e96-4dc3-8d3f-97a2f527cab6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:55.953447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9843d208-4adc-449c-8067-54e990c8c266 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:55.972253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:55.974124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:55.977594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:55.981712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 979/1936] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:55 2022] POST /identity/v3/users => generated 298 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:55.982707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] PATH_INFO: `/v3/users/9bd5b7c3f997479da0a6dd3def18cb1d` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:55.987300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9bd5b7c3f997479da0a6dd3def18cb1d)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:55.989623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:55.991935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9832adb-eba1-4838-926a-c19a56bd8bcf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:55.991935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9832adb-eba1-4838-926a-c19a56bd8bcf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:55.992157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9832adb-eba1-4838-926a-c19a56bd8bcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.069303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b9832adb-eba1-4838-926a-c19a56bd8bcf None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:56.071076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b9832adb-eba1-4838-926a-c19a56bd8bcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:56.089469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 980/1937] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:56.090700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9832adb-eba1-4838-926a-c19a56bd8bcf None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.119128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9832adb-eba1-4838-926a-c19a56bd8bcf None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:56.134881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675d88a-1f99-4eb2-b740-e43e8b9063de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:56.136207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675d88a-1f99-4eb2-b740-e43e8b9063de admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.136491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675d88a-1f99-4eb2-b740-e43e8b9063de admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.136767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7675d88a-1f99-4eb2-b740-e43e8b9063de admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.138260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675d88a-1f99-4eb2-b740-e43e8b9063de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:56.141078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7675d88a-1f99-4eb2-b740-e43e8b9063de admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:56.152646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 981/1938] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:56.153525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675d88a-1f99-4eb2-b740-e43e8b9063de admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.162608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9bd5b7c3f997479da0a6dd3def18cb1d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:56.167650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7675d88a-1f99-4eb2-b740-e43e8b9063de admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:56.174885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674152a-e72d-4a58-b880-7b4188b7e53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:56.175559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674152a-e72d-4a58-b880-7b4188b7e53f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.175890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674152a-e72d-4a58-b880-7b4188b7e53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.176238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6674152a-e72d-4a58-b880-7b4188b7e53f admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.177573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674152a-e72d-4a58-b880-7b4188b7e53f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:56.178168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674152a-e72d-4a58-b880-7b4188b7e53f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:56.181496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6674152a-e72d-4a58-b880-7b4188b7e53f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:56.183526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:56.186769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9bd5b7c3f997479da0a6dd3def18cb1d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:56.199368 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 982/1939] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:56.203809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674152a-e72d-4a58-b880-7b4188b7e53f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.218323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6674152a-e72d-4a58-b880-7b4188b7e53f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:56.227106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2c07-4e6c-46f6-8835-b5737b4c0835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:56.228004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2c07-4e6c-46f6-8835-b5737b4c0835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.228564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2c07-4e6c-46f6-8835-b5737b4c0835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.228848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58f2c07-4e6c-46f6-8835-b5737b4c0835 admin admin] PATH_INFO: `/v3/projects/5841a34081cb4056a278bbcb25d9c592/users/7aa2f2a9c22644d5b9035bc44222c6a8/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.248184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9bd5b7c3f997479da0a6dd3def18cb1d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:56.248724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2c07-4e6c-46f6-8835-b5737b4c0835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5841a34081cb4056a278bbcb25d9c592, user_id=7aa2f2a9c22644d5b9035bc44222c6a8, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:56.248926 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9bd5b7c3f997479da0a6dd3def18cb1d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:56.252771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58f2c07-4e6c-46f6-8835-b5737b4c0835 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:56.264729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9bd5b7c3f997479da0a6dd3def18cb1d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:56.267748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:56.280674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 983/1940] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:56 2022] PUT /identity/v3/projects/5841a34081cb4056a278bbcb25d9c592/users/7aa2f2a9c22644d5b9035bc44222c6a8/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:56.281644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2c07-4e6c-46f6-8835-b5737b4c0835 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.288346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b58f2c07-4e6c-46f6-8835-b5737b4c0835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:56.290767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 958/1941] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:01:55 2022] DELETE /identity/v3/users/9bd5b7c3f997479da0a6dd3def18cb1d => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:56.292396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.295690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:56.400673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58f2c07-4e6c-46f6-8835-b5737b4c0835 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:56.472106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0284c60a-9d86-4cf7-9665-d810ee5e4524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:56.472888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284c60a-9d86-4cf7-9665-d810ee5e4524 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.473212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284c60a-9d86-4cf7-9665-d810ee5e4524 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.473585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0284c60a-9d86-4cf7-9665-d810ee5e4524 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.474298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284c60a-9d86-4cf7-9665-d810ee5e4524 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:56.476344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0284c60a-9d86-4cf7-9665-d810ee5e4524 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:56.514517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc3c310-9cb7-49d5-88a5-7098ceea462b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:56.517823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 984/1942] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:56 2022] POST /identity/v3/projects => generated 391 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:56.529088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8a12a-4ba5-43ab-91a9-6dae06cd3a9b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.529429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8a12a-4ba5-43ab-91a9-6dae06cd3a9b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.529716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f8a12a-4ba5-43ab-91a9-6dae06cd3a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.573231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-95f8a12a-4ba5-43ab-91a9-6dae06cd3a9b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:56.575477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-95f8a12a-4ba5-43ab-91a9-6dae06cd3a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:56.662699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 985/1943] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:56.663472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8a12a-4ba5-43ab-91a9-6dae06cd3a9b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.673795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:56.683253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f8a12a-4ba5-43ab-91a9-6dae06cd3a9b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:56.693641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c5be5-d894-4f22-bd19-4861f1500b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:56.694732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c5be5-d894-4f22-bd19-4861f1500b02 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.695035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c5be5-d894-4f22-bd19-4861f1500b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.695702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c5be5-d894-4f22-bd19-4861f1500b02 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.696387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c5be5-d894-4f22-bd19-4861f1500b02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:56.700168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c5be5-d894-4f22-bd19-4861f1500b02 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:56.738259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 986/1944] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:56 2022] POST /identity/v3/users => generated 296 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:56.750120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c5be5-d894-4f22-bd19-4861f1500b02 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.754391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c43c5be5-d894-4f22-bd19-4861f1500b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:56.765404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd483c-31f5-4f1f-9952-a18c5ee0d75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:56.766116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd483c-31f5-4f1f-9952-a18c5ee0d75f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.766408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd483c-31f5-4f1f-9952-a18c5ee0d75f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.766799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fd483c-31f5-4f1f-9952-a18c5ee0d75f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.767621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd483c-31f5-4f1f-9952-a18c5ee0d75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:56.769658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fd483c-31f5-4f1f-9952-a18c5ee0d75f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:56.782692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 959/1945] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 491 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:56.783791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd483c-31f5-4f1f-9952-a18c5ee0d75f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.797593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fd483c-31f5-4f1f-9952-a18c5ee0d75f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:56.802955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994faba-e4fc-4e5b-a596-9a8229ce4e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:56.803803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994faba-e4fc-4e5b-a596-9a8229ce4e77 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.804091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994faba-e4fc-4e5b-a596-9a8229ce4e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.804370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7994faba-e4fc-4e5b-a596-9a8229ce4e77 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.805035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994faba-e4fc-4e5b-a596-9a8229ce4e77 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:56.807129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7994faba-e4fc-4e5b-a596-9a8229ce4e77 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:56.820676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 960/1946] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:56.821477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994faba-e4fc-4e5b-a596-9a8229ce4e77 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.833407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7994faba-e4fc-4e5b-a596-9a8229ce4e77 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:56.838759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bee297-7ec6-4228-990a-9fd3ede89d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:56.839444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bee297-7ec6-4228-990a-9fd3ede89d32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.839729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bee297-7ec6-4228-990a-9fd3ede89d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.840001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bee297-7ec6-4228-990a-9fd3ede89d32 admin admin] PATH_INFO: `/v3/projects/5841a34081cb4056a278bbcb25d9c592/users/7aa2f2a9c22644d5b9035bc44222c6a8/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.841390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c5be5-d894-4f22-bd19-4861f1500b02 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:56.841933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bee297-7ec6-4228-990a-9fd3ede89d32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5841a34081cb4056a278bbcb25d9c592, user_id=7aa2f2a9c22644d5b9035bc44222c6a8, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:56.844729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bee297-7ec6-4228-990a-9fd3ede89d32 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:56.846167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb754277-4209-422b-9e85-b52305c4629a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:56.846864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb754277-4209-422b-9e85-b52305c4629a service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.847577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb754277-4209-422b-9e85-b52305c4629a service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.848056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb754277-4209-422b-9e85-b52305c4629a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.859854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb754277-4209-422b-9e85-b52305c4629a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:56.869260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 961/1947] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:56 2022] PUT /identity/v3/projects/5841a34081cb4056a278bbcb25d9c592/users/7aa2f2a9c22644d5b9035bc44222c6a8/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:56.869260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bee297-7ec6-4228-990a-9fd3ede89d32 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:56.869573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb754277-4209-422b-9e85-b52305c4629a service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:56.878266 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 987/1948] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:56.882009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8eb46-1041-425b-9368-933740d16437 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:56.883042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8eb46-1041-425b-9368-933740d16437 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:56.883812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d8eb46-1041-425b-9368-933740d16437 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:56.950269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d7d8eb46-1041-425b-9368-933740d16437 None None] MFA Rules not processed for user `7aa2f2a9c22644d5b9035bc44222c6a8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:56.951646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d7d8eb46-1041-425b-9368-933740d16437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:56.998726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bee297-7ec6-4228-990a-9fd3ede89d32 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:57.011255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4714f35-c836-4ed7-87b4-cf79c42038af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:57.013160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4714f35-c836-4ed7-87b4-cf79c42038af admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.013465 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4714f35-c836-4ed7-87b4-cf79c42038af admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.013742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4714f35-c836-4ed7-87b4-cf79c42038af admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.015716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4714f35-c836-4ed7-87b4-cf79c42038af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:57.020705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4714f35-c836-4ed7-87b4-cf79c42038af admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:57.047568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 962/1949] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:56 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:57.049619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4714f35-c836-4ed7-87b4-cf79c42038af admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:57.071431 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4714f35-c836-4ed7-87b4-cf79c42038af admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:57.085669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9568f4ad-0072-4e74-a555-3c5d928a22d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:57.086570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568f4ad-0072-4e74-a555-3c5d928a22d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.086691 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568f4ad-0072-4e74-a555-3c5d928a22d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.086972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9568f4ad-0072-4e74-a555-3c5d928a22d8 admin admin] PATH_INFO: `/v3/projects/58d0385926c54d0b8d21fc98cfaa21cf/users/912fdfe24b9349eb847eb62999475163/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.119084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:57.122917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568f4ad-0072-4e74-a555-3c5d928a22d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58d0385926c54d0b8d21fc98cfaa21cf, user_id=912fdfe24b9349eb847eb62999475163, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:57.125650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9568f4ad-0072-4e74-a555-3c5d928a22d8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:57.132375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 988/1950] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:57.138038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8eb46-1041-425b-9368-933740d16437 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:57.139849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d7d8eb46-1041-425b-9368-933740d16437 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:57.158388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 963/1951] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:57 2022] PUT /identity/v3/projects/58d0385926c54d0b8d21fc98cfaa21cf/users/912fdfe24b9349eb847eb62999475163/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:57.176933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4c260-fc5f-4eeb-820c-2ee7a7bf13ab None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.177328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4c260-fc5f-4eeb-820c-2ee7a7bf13ab None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.177740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4c260-fc5f-4eeb-820c-2ee7a7bf13ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.237517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d1b4c260-fc5f-4eeb-820c-2ee7a7bf13ab None None] MFA Rules not processed for user `912fdfe24b9349eb847eb62999475163`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:57.239191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d1b4c260-fc5f-4eeb-820c-2ee7a7bf13ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:57.275002 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d8eb46-1041-425b-9368-933740d16437 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:57.322667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:57.370015 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 964/1952] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:57 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:57.371608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4c260-fc5f-4eeb-820c-2ee7a7bf13ab None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:57.373300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d1b4c260-fc5f-4eeb-820c-2ee7a7bf13ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:57.381785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d83b85-91d7-4632-955c-9cc50b823fbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:57.384061 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d83b85-91d7-4632-955c-9cc50b823fbe service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.384352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d83b85-91d7-4632-955c-9cc50b823fbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.384633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d83b85-91d7-4632-955c-9cc50b823fbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.462716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4c260-fc5f-4eeb-820c-2ee7a7bf13ab None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:57.469429 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:57.470749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.471210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.471319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] PATH_INFO: `/v3/projects/8efaeab9973245179f4d3ebc6e8aaf1b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.472442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8efaeab9973245179f4d3ebc6e8aaf1b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:57.474824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:57.481610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d83b85-91d7-4632-955c-9cc50b823fbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:57.483636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d83b85-91d7-4632-955c-9cc50b823fbe service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:57.497491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 989/1953] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:57.504767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d83b85-91d7-4632-955c-9cc50b823fbe service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:57.518779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d83b85-91d7-4632-955c-9cc50b823fbe service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:57.526063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1bb3-3f63-4981-b981-86ceb4cd3fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:57.527307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1bb3-3f63-4981-b981-86ceb4cd3fd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.527738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1bb3-3f63-4981-b981-86ceb4cd3fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.528011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc1bb3-3f63-4981-b981-86ceb4cd3fd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.529321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc1bb3-3f63-4981-b981-86ceb4cd3fd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:57.533122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc1bb3-3f63-4981-b981-86ceb4cd3fd5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:57.606790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 990/1954] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:57 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:57.609322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8efaeab9973245179f4d3ebc6e8aaf1b', 'request_id': 'req-2809514c-3752-407a-b8bb-70498d8427f2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:57.612637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1bb3-3f63-4981-b981-86ceb4cd3fd5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:57.630207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8efaeab9973245179f4d3ebc6e8aaf1b', 'request_id': 'req-2809514c-3752-407a-b8bb-70498d8427f2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:57.645777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8efaeab9973245179f4d3ebc6e8aaf1b', 'request_id': 'req-2809514c-3752-407a-b8bb-70498d8427f2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:57.679458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] The token cache is being invalidate because project 8efaeab9973245179f4d3ebc6e8aaf1b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:01:57.679804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:57.682992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:01:57.695964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 965/1955] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:01:57 2022] DELETE /identity/v3/projects/8efaeab9973245179f4d3ebc6e8aaf1b => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:57.700485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:57.703982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:57.729588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:57.753453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc1bb3-3f63-4981-b981-86ceb4cd3fd5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:57.762439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf27f49-a1cf-4ac7-bc15-e28588d30fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:57.763178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf27f49-a1cf-4ac7-bc15-e28588d30fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.763456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf27f49-a1cf-4ac7-bc15-e28588d30fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.763733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf27f49-a1cf-4ac7-bc15-e28588d30fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.764379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf27f49-a1cf-4ac7-bc15-e28588d30fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:57.767938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf27f49-a1cf-4ac7-bc15-e28588d30fb3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:57.808834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2809514c-3752-407a-b8bb-70498d8427f2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:57.838706 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 991/1956] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:57 2022] POST /identity/v3/users => generated 301 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:57.844319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf54d33-a6a8-4c77-af55-3a6ed14cab0e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.844811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf54d33-a6a8-4c77-af55-3a6ed14cab0e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.845253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf54d33-a6a8-4c77-af55-3a6ed14cab0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.891345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389aa116-d9b9-44ea-8a40-700d2c01b52f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:57.892267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389aa116-d9b9-44ea-8a40-700d2c01b52f service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.892529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389aa116-d9b9-44ea-8a40-700d2c01b52f service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.892901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389aa116-d9b9-44ea-8a40-700d2c01b52f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.896283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-389aa116-d9b9-44ea-8a40-700d2c01b52f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:57.899960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fcf54d33-a6a8-4c77-af55-3a6ed14cab0e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:57.901277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fcf54d33-a6a8-4c77-af55-3a6ed14cab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:57.914281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 992/1957] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:57.917233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf54d33-a6a8-4c77-af55-3a6ed14cab0e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:57.920015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fcf54d33-a6a8-4c77-af55-3a6ed14cab0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:57.945382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf54d33-a6a8-4c77-af55-3a6ed14cab0e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:57.963985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7d9fa3-991a-447e-8008-6ada808825b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:57.965937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d9fa3-991a-447e-8008-6ada808825b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.966660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d9fa3-991a-447e-8008-6ada808825b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.967139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7d9fa3-991a-447e-8008-6ada808825b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:57.969221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d9fa3-991a-447e-8008-6ada808825b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:57.972852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7d9fa3-991a-447e-8008-6ada808825b6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:57.996338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 993/1958] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:57 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:57.998291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e167b-bd4c-4ed6-ad6f-2f76984539f8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:57.998816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e167b-bd4c-4ed6-ad6f-2f76984539f8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:57.999341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9e167b-bd4c-4ed6-ad6f-2f76984539f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.021218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9e9e167b-bd4c-4ed6-ad6f-2f76984539f8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:58.022918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9e9e167b-bd4c-4ed6-ad6f-2f76984539f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:58.023739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389aa116-d9b9-44ea-8a40-700d2c01b52f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.026504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389aa116-d9b9-44ea-8a40-700d2c01b52f service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.034919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 994/1959] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:58.034919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e167b-bd4c-4ed6-ad6f-2f76984539f8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.043365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9e167b-bd4c-4ed6-ad6f-2f76984539f8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.043746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 966/1960] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:57 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:58.044844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389aa116-d9b9-44ea-8a40-700d2c01b52f service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.049036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97950888-1c6e-4ba2-815f-948b4365fe4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.049712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97950888-1c6e-4ba2-815f-948b4365fe4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.050035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97950888-1c6e-4ba2-815f-948b4365fe4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.050343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97950888-1c6e-4ba2-815f-948b4365fe4b admin admin] PATH_INFO: `/v3/projects/6d3da5f651a54857a1da99125487d3ef/users/c8b432a53be2419997acacfc9a7d071d/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.070129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97950888-1c6e-4ba2-815f-948b4365fe4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3da5f651a54857a1da99125487d3ef, user_id=c8b432a53be2419997acacfc9a7d071d, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.072958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97950888-1c6e-4ba2-815f-948b4365fe4b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.082428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389aa116-d9b9-44ea-8a40-700d2c01b52f service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.092617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:58.105080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 995/1961] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:58 2022] PUT /identity/v3/projects/6d3da5f651a54857a1da99125487d3ef/users/c8b432a53be2419997acacfc9a7d071d/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:58.109780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97950888-1c6e-4ba2-815f-948b4365fe4b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.121265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af71bdd-10b2-40fd-b643-22c3244563ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.122541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af71bdd-10b2-40fd-b643-22c3244563ff admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.123176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af71bdd-10b2-40fd-b643-22c3244563ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.124504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af71bdd-10b2-40fd-b643-22c3244563ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.125656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af71bdd-10b2-40fd-b643-22c3244563ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:01:58.126204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af71bdd-10b2-40fd-b643-22c3244563ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.130420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af71bdd-10b2-40fd-b643-22c3244563ff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.154340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 967/1962] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:01:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:01:58.158831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af71bdd-10b2-40fd-b643-22c3244563ff admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.184128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af71bdd-10b2-40fd-b643-22c3244563ff admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.250786 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97950888-1c6e-4ba2-815f-948b4365fe4b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.264262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b871f-1940-424a-b8c6-f4e199530194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.264839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b871f-1940-424a-b8c6-f4e199530194 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.265115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b871f-1940-424a-b8c6-f4e199530194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.265388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b871f-1940-424a-b8c6-f4e199530194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.266059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b871f-1940-424a-b8c6-f4e199530194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.267715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257b871f-1940-424a-b8c6-f4e199530194 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.292994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 996/1963] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:58.293991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b871f-1940-424a-b8c6-f4e199530194 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.318797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b871f-1940-424a-b8c6-f4e199530194 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.326590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81099627-5e55-44b4-b0a2-eb4e693be3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.327408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81099627-5e55-44b4-b0a2-eb4e693be3b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.327730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81099627-5e55-44b4-b0a2-eb4e693be3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.328027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81099627-5e55-44b4-b0a2-eb4e693be3b7 admin admin] PATH_INFO: `/v3/projects/6d3da5f651a54857a1da99125487d3ef/users/c8b432a53be2419997acacfc9a7d071d/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.331385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81099627-5e55-44b4-b0a2-eb4e693be3b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d3da5f651a54857a1da99125487d3ef, user_id=c8b432a53be2419997acacfc9a7d071d, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.335317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81099627-5e55-44b4-b0a2-eb4e693be3b7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.339905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d68ce-6d84-4895-8a99-fec35bb504bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.341076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d68ce-6d84-4895-8a99-fec35bb504bc admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.341830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d68ce-6d84-4895-8a99-fec35bb504bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.342497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d68ce-6d84-4895-8a99-fec35bb504bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.343995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d68ce-6d84-4895-8a99-fec35bb504bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.348097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d68ce-6d84-4895-8a99-fec35bb504bc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.351693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 997/1964] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:58 2022] PUT /identity/v3/projects/6d3da5f651a54857a1da99125487d3ef/users/c8b432a53be2419997acacfc9a7d071d/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:58.365201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858fe52-1123-4236-bd00-f4a4861dfefa None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.366311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858fe52-1123-4236-bd00-f4a4861dfefa None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.368272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d858fe52-1123-4236-bd00-f4a4861dfefa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.398763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:58.420014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 968/1965] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:58 2022] POST /identity/v3/projects => generated 373 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:58.424462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d68ce-6d84-4895-8a99-fec35bb504bc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.454856 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d858fe52-1123-4236-bd00-f4a4861dfefa None None] MFA Rules not processed for user `c8b432a53be2419997acacfc9a7d071d`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:58.456679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d858fe52-1123-4236-bd00-f4a4861dfefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:58.523771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d68ce-6d84-4895-8a99-fec35bb504bc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.531637 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f46779-1956-4963-9f16-1b844b20faf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.533516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f46779-1956-4963-9f16-1b844b20faf8 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.534007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f46779-1956-4963-9f16-1b844b20faf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.534440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f46779-1956-4963-9f16-1b844b20faf8 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.535746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f46779-1956-4963-9f16-1b844b20faf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.538542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f46779-1956-4963-9f16-1b844b20faf8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.589938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 969/1966] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:58 2022] POST /identity/v3/users => generated 287 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:58.615411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f46779-1956-4963-9f16-1b844b20faf8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.635569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 998/1967] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:58 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:01:58.641450 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f46779-1956-4963-9f16-1b844b20faf8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.649035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858fe52-1123-4236-bd00-f4a4861dfefa None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.651488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d858fe52-1123-4236-bd00-f4a4861dfefa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:58.657827 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f6ed2-5383-48ae-b1ed-2c4293991444 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.658527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f6ed2-5383-48ae-b1ed-2c4293991444 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.658811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f6ed2-5383-48ae-b1ed-2c4293991444 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.659085 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2f6ed2-5383-48ae-b1ed-2c4293991444 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.660454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f6ed2-5383-48ae-b1ed-2c4293991444 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.661509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce2f6ed2-5383-48ae-b1ed-2c4293991444 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.690504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 970/1968] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:58.692398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f6ed2-5383-48ae-b1ed-2c4293991444 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.710946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2f6ed2-5383-48ae-b1ed-2c4293991444 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.721090 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f929a09-3a5b-4a43-9bee-9e3b1d6e50c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.721906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f929a09-3a5b-4a43-9bee-9e3b1d6e50c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.722320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f929a09-3a5b-4a43-9bee-9e3b1d6e50c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.722698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f929a09-3a5b-4a43-9bee-9e3b1d6e50c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.724489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f929a09-3a5b-4a43-9bee-9e3b1d6e50c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.726455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f929a09-3a5b-4a43-9bee-9e3b1d6e50c3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.749941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 971/1969] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:58.751450 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f929a09-3a5b-4a43-9bee-9e3b1d6e50c3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.767712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f929a09-3a5b-4a43-9bee-9e3b1d6e50c3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.774037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292f2be-0080-4600-bf1d-90bdb7648538 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.774933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f2be-0080-4600-bf1d-90bdb7648538 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.775324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f2be-0080-4600-bf1d-90bdb7648538 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.775698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292f2be-0080-4600-bf1d-90bdb7648538 admin admin] PATH_INFO: `/v3/projects/44b2db2ba59f4725b4319ef2cddb5f66/users/33bcb710d0004a5a8b1256cf4e303f10/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.792456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d858fe52-1123-4236-bd00-f4a4861dfefa None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:58.794491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292f2be-0080-4600-bf1d-90bdb7648538 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b2db2ba59f4725b4319ef2cddb5f66, user_id=33bcb710d0004a5a8b1256cf4e303f10, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.797598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292f2be-0080-4600-bf1d-90bdb7648538 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.806285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00714b08-b5ce-4c8f-8a31-84a248ac2e9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:58.807195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00714b08-b5ce-4c8f-8a31-84a248ac2e9d service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.807649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00714b08-b5ce-4c8f-8a31-84a248ac2e9d service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.808442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00714b08-b5ce-4c8f-8a31-84a248ac2e9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.821320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00714b08-b5ce-4c8f-8a31-84a248ac2e9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:58.824329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00714b08-b5ce-4c8f-8a31-84a248ac2e9d service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:58.831407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 972/1970] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:58 2022] PUT /identity/v3/projects/44b2db2ba59f4725b4319ef2cddb5f66/users/33bcb710d0004a5a8b1256cf4e303f10/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:58.843740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8c6bc-b7df-437f-b234-883d37d809b5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:58.844327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8c6bc-b7df-437f-b234-883d37d809b5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:58.844771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f8c6bc-b7df-437f-b234-883d37d809b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:58.860718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 999/1971] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:58 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:01:58.944055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f9f8c6bc-b7df-437f-b234-883d37d809b5 None None] MFA Rules not processed for user `33bcb710d0004a5a8b1256cf4e303f10`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:58.944716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f9f8c6bc-b7df-437f-b234-883d37d809b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:58.979715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00714b08-b5ce-4c8f-8a31-84a248ac2e9d service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:58.992283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00714b08-b5ce-4c8f-8a31-84a248ac2e9d service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:59.012273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:59.094577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 973/1972] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:01:59.099248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8c6bc-b7df-437f-b234-883d37d809b5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:59.119014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d3f2b-4d9e-473d-9758-1acd94ab685e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:59.119672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d3f2b-4d9e-473d-9758-1acd94ab685e service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:59.119948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d3f2b-4d9e-473d-9758-1acd94ab685e service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:59.120219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50d3f2b-4d9e-473d-9758-1acd94ab685e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:59.122684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f50d3f2b-4d9e-473d-9758-1acd94ab685e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:01:59.146818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:01:59.225241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f8c6bc-b7df-437f-b234-883d37d809b5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:59.232732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d21c6-a581-42da-9a6b-a739dd3d8747 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:59.233363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d21c6-a581-42da-9a6b-a739dd3d8747 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:59.233641 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d21c6-a581-42da-9a6b-a739dd3d8747 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:59.233909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029d21c6-a581-42da-9a6b-a739dd3d8747 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:59.239036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d3f2b-4d9e-473d-9758-1acd94ab685e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:59.240717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50d3f2b-4d9e-473d-9758-1acd94ab685e service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:59.240833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d21c6-a581-42da-9a6b-a739dd3d8747 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:59.242906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029d21c6-a581-42da-9a6b-a739dd3d8747 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:59.252855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1000/1973] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:01:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:01:59.290977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d3f2b-4d9e-473d-9758-1acd94ab685e service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:59.293632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 974/1974] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:01:59 2022] POST /identity/v3/projects => generated 373 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:59.307391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50d3f2b-4d9e-473d-9758-1acd94ab685e service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:59.413035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebb14d-f4dc-440f-8046-de55b9508345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:59.415949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebb14d-f4dc-440f-8046-de55b9508345 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:59.417700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebb14d-f4dc-440f-8046-de55b9508345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:59.418047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ebb14d-f4dc-440f-8046-de55b9508345 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:59.421235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebb14d-f4dc-440f-8046-de55b9508345 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:59.423262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ebb14d-f4dc-440f-8046-de55b9508345 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:59.485637 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d21c6-a581-42da-9a6b-a739dd3d8747 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:59.489715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1001/1975] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:01:59 2022] POST /identity/v3/users => generated 293 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:01:59.504315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029d21c6-a581-42da-9a6b-a739dd3d8747 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:59.516449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ec9fb-9006-4207-bfbd-1fd0c0f122fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:59.517112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ec9fb-9006-4207-bfbd-1fd0c0f122fb admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:59.517993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ec9fb-9006-4207-bfbd-1fd0c0f122fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:59.518355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ec9fb-9006-4207-bfbd-1fd0c0f122fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:59.519421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ec9fb-9006-4207-bfbd-1fd0c0f122fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:59.521904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ec9fb-9006-4207-bfbd-1fd0c0f122fb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:59.545979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 975/1976] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:01:59.551135 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebb14d-f4dc-440f-8046-de55b9508345 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:59.601427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ebb14d-f4dc-440f-8046-de55b9508345 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:59.618884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4ae29-6f5a-480b-8a1a-7bcafec16e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:59.619743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4ae29-6f5a-480b-8a1a-7bcafec16e1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:59.619931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4ae29-6f5a-480b-8a1a-7bcafec16e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:59.622616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4ae29-6f5a-480b-8a1a-7bcafec16e1f admin admin] PATH_INFO: `/v3/projects/5c02a16e2b1c4c2fa04075f182d44bac/users/3412679ed4124c36a33ac05c53a33a23/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:59.686114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4ae29-6f5a-480b-8a1a-7bcafec16e1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c02a16e2b1c4c2fa04075f182d44bac, user_id=3412679ed4124c36a33ac05c53a33a23, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:59.692864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b4ae29-6f5a-480b-8a1a-7bcafec16e1f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:59.733681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1002/1977] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:59 2022] PUT /identity/v3/projects/5c02a16e2b1c4c2fa04075f182d44bac/users/3412679ed4124c36a33ac05c53a33a23/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:01:59.734761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ec9fb-9006-4207-bfbd-1fd0c0f122fb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:59.748735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ec9fb-9006-4207-bfbd-1fd0c0f122fb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:59.860937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d92677-cfce-436f-b12f-838135f90906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:59.861715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d92677-cfce-436f-b12f-838135f90906 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:59.862020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d92677-cfce-436f-b12f-838135f90906 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:59.862385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d92677-cfce-436f-b12f-838135f90906 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:59.862988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d92677-cfce-436f-b12f-838135f90906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:59.864974 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d92677-cfce-436f-b12f-838135f90906 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:59.879162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 976/1978] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:01:59.880482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4ae29-6f5a-480b-8a1a-7bcafec16e1f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:01:59.894188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4ae29-6f5a-480b-8a1a-7bcafec16e1f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:01:59.899510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3595409-b8ff-4956-a3d5-1b2caf5bcaf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:01:59.900365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3595409-b8ff-4956-a3d5-1b2caf5bcaf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:59.900652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3595409-b8ff-4956-a3d5-1b2caf5bcaf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:59.900929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3595409-b8ff-4956-a3d5-1b2caf5bcaf5 admin admin] PATH_INFO: `/v3/projects/5c02a16e2b1c4c2fa04075f182d44bac/users/3412679ed4124c36a33ac05c53a33a23/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:59.902892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3595409-b8ff-4956-a3d5-1b2caf5bcaf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c02a16e2b1c4c2fa04075f182d44bac, user_id=3412679ed4124c36a33ac05c53a33a23, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:01:59.905540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3595409-b8ff-4956-a3d5-1b2caf5bcaf5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:01:59.917228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1003/1979] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:01:59 2022] PUT /identity/v3/projects/5c02a16e2b1c4c2fa04075f182d44bac/users/3412679ed4124c36a33ac05c53a33a23/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:01:59.927770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50594f-92af-45a0-b627-688ccb31f186 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:01:59.928056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50594f-92af-45a0-b627-688ccb31f186 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:01:59.928333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd50594f-92af-45a0-b627-688ccb31f186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:01:59.957545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cd50594f-92af-45a0-b627-688ccb31f186 None None] MFA Rules not processed for user `3412679ed4124c36a33ac05c53a33a23`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:01:59.959128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cd50594f-92af-45a0-b627-688ccb31f186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:00.059442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 977/1980] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:01:59 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:00.070582 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3595409-b8ff-4956-a3d5-1b2caf5bcaf5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:00.102722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:00.198051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3595409-b8ff-4956-a3d5-1b2caf5bcaf5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:00.203590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16c0bea-5eae-4269-98a6-da18e856f6b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:00.204233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c0bea-5eae-4269-98a6-da18e856f6b0 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:00.204515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c0bea-5eae-4269-98a6-da18e856f6b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:00.204791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16c0bea-5eae-4269-98a6-da18e856f6b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:00.315896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c0bea-5eae-4269-98a6-da18e856f6b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:00.317698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16c0bea-5eae-4269-98a6-da18e856f6b0 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:00.329987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1004/1981] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:01.594156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50594f-92af-45a0-b627-688ccb31f186 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:01.596567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cd50594f-92af-45a0-b627-688ccb31f186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:01.627352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:01.672395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd50594f-92af-45a0-b627-688ccb31f186 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:01.686692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:01.688323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:01.688858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:01.689344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] PATH_INFO: `/v3/users/c704810fc4c14d04a70c5cce5a30cce5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:01.690830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c704810fc4c14d04a70c5cce5a30cce5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:01.693277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:01.748094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c704810fc4c14d04a70c5cce5a30cce5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:01.757683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c704810fc4c14d04a70c5cce5a30cce5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:01.773215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c704810fc4c14d04a70c5cce5a30cce5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:01.773942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c704810fc4c14d04a70c5cce5a30cce5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:01.781769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c704810fc4c14d04a70c5cce5a30cce5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:01.789159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 978/1982] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:01 2022] DELETE /identity/v3/users/c704810fc4c14d04a70c5cce5a30cce5 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:01.793522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b0e2b-4397-4498-9879-9733d9b901dd None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:01.794013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b0e2b-4397-4498-9879-9733d9b901dd None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:01.794124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34b0e2b-4397-4498-9879-9733d9b901dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:01.827424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d34b0e2b-4397-4498-9879-9733d9b901dd None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:01.829440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d34b0e2b-4397-4498-9879-9733d9b901dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:01.918475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1005/1983] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:01.932103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:01.933813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:02.006023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e23d591-ecad-4237-ac02-2a2635f01fa3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:02.010024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb8449-e9f1-4634-bae5-3306e1d4aaad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:02.010672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb8449-e9f1-4634-bae5-3306e1d4aaad service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:02.010949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb8449-e9f1-4634-bae5-3306e1d4aaad service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:02.011248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eb8449-e9f1-4634-bae5-3306e1d4aaad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:02.020702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb8449-e9f1-4634-bae5-3306e1d4aaad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:02.022171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eb8449-e9f1-4634-bae5-3306e1d4aaad service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:02.032199 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 979/1984] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:02.328683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b0e2b-4397-4498-9879-9733d9b901dd None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:02.330352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d34b0e2b-4397-4498-9879-9733d9b901dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:02.345219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34b0e2b-4397-4498-9879-9733d9b901dd None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:02.351090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:02.351856 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:02.352200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:02.352606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] PATH_INFO: `/v3/projects/29798d0dec39434abd773f967881f7a5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:02.353806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=29798d0dec39434abd773f967881f7a5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:02.356102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:02.405799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29798d0dec39434abd773f967881f7a5', 'request_id': 'req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.412241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29798d0dec39434abd773f967881f7a5', 'request_id': 'req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.417782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29798d0dec39434abd773f967881f7a5', 'request_id': 'req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.420357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:02.434223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] The token cache is being invalidate because project 29798d0dec39434abd773f967881f7a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:02.434747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.435518 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.437025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1006/1985] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:02 2022] DELETE /identity/v3/projects/29798d0dec39434abd773f967881f7a5 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:02.441205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb8449-e9f1-4634-bae5-3306e1d4aaad service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:02.451117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eb8449-e9f1-4634-bae5-3306e1d4aaad service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:02.494014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:02.511566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:02.512224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:02.512519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:02.512789 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] PATH_INFO: `/v3/users/7aa2f2a9c22644d5b9035bc44222c6a8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:02.514057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aa2f2a9c22644d5b9035bc44222c6a8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:02.516057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:02.568755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aa2f2a9c22644d5b9035bc44222c6a8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.577386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aa2f2a9c22644d5b9035bc44222c6a8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.590576 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aa2f2a9c22644d5b9035bc44222c6a8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.591318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aa2f2a9c22644d5b9035bc44222c6a8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.607768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aa2f2a9c22644d5b9035bc44222c6a8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:02.616016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 980/1986] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:02 2022] DELETE /identity/v3/users/7aa2f2a9c22644d5b9035bc44222c6a8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:02.887797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:02.889610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:02.902858 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42eb357f-fe4b-4033-ad10-2bd03e2a6b0e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:02.903159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:02.904612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:02.977264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:02.978103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:02.978564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:02.979057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] PATH_INFO: `/v3/users/912fdfe24b9349eb847eb62999475163` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:02.980286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=912fdfe24b9349eb847eb62999475163)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:02.985738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.033401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48099ef-dfd8-4f75-a472-9b7d2889aed0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.038641 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.039304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.039582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.039857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] PATH_INFO: `/v3/projects/5841a34081cb4056a278bbcb25d9c592` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.041083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5841a34081cb4056a278bbcb25d9c592)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.043066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.051299 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '912fdfe24b9349eb847eb62999475163'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.051968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '912fdfe24b9349eb847eb62999475163'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.063550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '912fdfe24b9349eb847eb62999475163'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.082200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '912fdfe24b9349eb847eb62999475163'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.084239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '912fdfe24b9349eb847eb62999475163'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.108820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1007/1987] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:02 2022] DELETE /identity/v3/users/912fdfe24b9349eb847eb62999475163 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:03.110945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:03.112611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:03.116283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5841a34081cb4056a278bbcb25d9c592', 'request_id': 'req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.122890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5841a34081cb4056a278bbcb25d9c592', 'request_id': 'req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.135908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5841a34081cb4056a278bbcb25d9c592', 'request_id': 'req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.138904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:03.155057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] The token cache is being invalidate because project 5841a34081cb4056a278bbcb25d9c592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:03.155524 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.156685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97d11f67-8e14-4a2f-a95e-c5fbfde7c217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.158867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 981/1988] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:02 2022] DELETE /identity/v3/projects/5841a34081cb4056a278bbcb25d9c592 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:03.164172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617ba8-1e0e-44c2-aed8-7b798f46bfaa None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.164172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617ba8-1e0e-44c2-aed8-7b798f46bfaa None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.164172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61617ba8-1e0e-44c2-aed8-7b798f46bfaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.182991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-61617ba8-1e0e-44c2-aed8-7b798f46bfaa None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:03.184198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-61617ba8-1e0e-44c2-aed8-7b798f46bfaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:03.200332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad7df14-1519-49cd-91b0-5f4a4efe06b3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.205031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.205688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.206001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.206302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] PATH_INFO: `/v3/users/33bcb710d0004a5a8b1256cf4e303f10` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.207449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33bcb710d0004a5a8b1256cf4e303f10)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.209509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.223176 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:03.249014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 982/1989] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:03.249014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0d07f-051d-40cb-b3dc-75a0e4b9c28e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.249014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0d07f-051d-40cb-b3dc-75a0e4b9c28e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.249014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0d07f-051d-40cb-b3dc-75a0e4b9c28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.262543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33bcb710d0004a5a8b1256cf4e303f10'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.263229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33bcb710d0004a5a8b1256cf4e303f10'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.269385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1ed0d07f-051d-40cb-b3dc-75a0e4b9c28e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:03.270915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1ed0d07f-051d-40cb-b3dc-75a0e4b9c28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:03.271555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33bcb710d0004a5a8b1256cf4e303f10'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.279057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 983/1990] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:03.279925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0d07f-051d-40cb-b3dc-75a0e4b9c28e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:03.281202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1ed0d07f-051d-40cb-b3dc-75a0e4b9c28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:03.291727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33bcb710d0004a5a8b1256cf4e303f10'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.292976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-af7c94c6-9681-45ba-9810-0c1eeb573807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33bcb710d0004a5a8b1256cf4e303f10'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:03.307827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1008/1991] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:03 2022] DELETE /identity/v3/users/33bcb710d0004a5a8b1256cf4e303f10 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:03.309320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b6d3e-bc17-486e-a6f3-e03a43aa137c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.309613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b6d3e-bc17-486e-a6f3-e03a43aa137c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.309921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755b6d3e-bc17-486e-a6f3-e03a43aa137c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.330899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-755b6d3e-bc17-486e-a6f3-e03a43aa137c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:03.332138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-755b6d3e-bc17-486e-a6f3-e03a43aa137c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:03.343120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0d07f-051d-40cb-b3dc-75a0e4b9c28e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.393123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec1b85-9e30-4e1f-b5b2-e90b88f20e10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.393732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1009/1992] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:03.393861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec1b85-9e30-4e1f-b5b2-e90b88f20e10 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.394030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec1b85-9e30-4e1f-b5b2-e90b88f20e10 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.394302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ec1b85-9e30-4e1f-b5b2-e90b88f20e10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.395294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c14bf-57b9-40b8-b4ab-01126481d36d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.395413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c14bf-57b9-40b8-b4ab-01126481d36d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.395650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6c14bf-57b9-40b8-b4ab-01126481d36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.398763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e2ec1b85-9e30-4e1f-b5b2-e90b88f20e10 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:03.411474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec1b85-9e30-4e1f-b5b2-e90b88f20e10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.412939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ec1b85-9e30-4e1f-b5b2-e90b88f20e10 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.416041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2b6c14bf-57b9-40b8-b4ab-01126481d36d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:03.417233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2b6c14bf-57b9-40b8-b4ab-01126481d36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:03.422527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 984/1993] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:03.423312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec1b85-9e30-4e1f-b5b2-e90b88f20e10 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:03.425076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1010/1994] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:03.435050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c14bf-57b9-40b8-b4ab-01126481d36d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:03.436442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2b6c14bf-57b9-40b8-b4ab-01126481d36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:03.436840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ec1b85-9e30-4e1f-b5b2-e90b88f20e10 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.441612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8afe84-fe70-4e11-84f1-ef325ba926de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.442439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8afe84-fe70-4e11-84f1-ef325ba926de admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.442664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8afe84-fe70-4e11-84f1-ef325ba926de admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.443144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8afe84-fe70-4e11-84f1-ef325ba926de admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.443568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8afe84-fe70-4e11-84f1-ef325ba926de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:03.443869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8afe84-fe70-4e11-84f1-ef325ba926de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.445469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8afe84-fe70-4e11-84f1-ef325ba926de admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.452054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6c14bf-57b9-40b8-b4ab-01126481d36d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.457295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b34ba5-09c2-4441-a08e-7388275250b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.458216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 985/1995] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:03.458913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b34ba5-09c2-4441-a08e-7388275250b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.459268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b34ba5-09c2-4441-a08e-7388275250b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.459877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b34ba5-09c2-4441-a08e-7388275250b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.462393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8afe84-fe70-4e11-84f1-ef325ba926de admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:03.470418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b34ba5-09c2-4441-a08e-7388275250b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.471948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b34ba5-09c2-4441-a08e-7388275250b2 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.472361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8afe84-fe70-4e11-84f1-ef325ba926de admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.477527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437698d6-e49d-4023-a4c6-3309f109f2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.479469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437698d6-e49d-4023-a4c6-3309f109f2fe admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.480244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437698d6-e49d-4023-a4c6-3309f109f2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.480377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437698d6-e49d-4023-a4c6-3309f109f2fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.481163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437698d6-e49d-4023-a4c6-3309f109f2fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.483752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437698d6-e49d-4023-a4c6-3309f109f2fe admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.485191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1011/1996] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:02:03.519244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 986/1997] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:03 2022] POST /identity/v3/projects => generated 381 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:03.521004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b34ba5-09c2-4441-a08e-7388275250b2 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:03.532346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b34ba5-09c2-4441-a08e-7388275250b2 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.635884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039d90f-e0aa-4e4c-969a-fdb38d0ac57f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.636611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039d90f-e0aa-4e4c-969a-fdb38d0ac57f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.636981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039d90f-e0aa-4e4c-969a-fdb38d0ac57f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.637327 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a039d90f-e0aa-4e4c-969a-fdb38d0ac57f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.638177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039d90f-e0aa-4e4c-969a-fdb38d0ac57f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.640147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a039d90f-e0aa-4e4c-969a-fdb38d0ac57f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.681686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1012/1998] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:03 2022] POST /identity/v3/users => generated 291 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:03.686009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437698d6-e49d-4023-a4c6-3309f109f2fe admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:03.698711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437698d6-e49d-4023-a4c6-3309f109f2fe admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.704570 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d95ee-1568-4084-ad2e-0a5abcb3709f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.705286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d95ee-1568-4084-ad2e-0a5abcb3709f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.705635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d95ee-1568-4084-ad2e-0a5abcb3709f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.705974 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5d95ee-1568-4084-ad2e-0a5abcb3709f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.707248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d95ee-1568-4084-ad2e-0a5abcb3709f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.709129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea5d95ee-1568-4084-ad2e-0a5abcb3709f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.716201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:03.726017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039d90f-e0aa-4e4c-969a-fdb38d0ac57f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:03.732393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=87,keystone:DELETE=9,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:03.734932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 987/1999] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:03.740635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a039d90f-e0aa-4e4c-969a-fdb38d0ac57f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.748504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e71f3-c132-487e-879b-43b11f84e202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.749296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e71f3-c132-487e-879b-43b11f84e202 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.749938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e71f3-c132-487e-879b-43b11f84e202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.750573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e71f3-c132-487e-879b-43b11f84e202 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.751784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e71f3-c132-487e-879b-43b11f84e202 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.754020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e71f3-c132-487e-879b-43b11f84e202 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.770244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1013/2000] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:03.771546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d95ee-1568-4084-ad2e-0a5abcb3709f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:03.790701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5d95ee-1568-4084-ad2e-0a5abcb3709f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:03.806243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959b045-45ad-47b5-8c5b-7fb0c605cf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:03.807336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959b045-45ad-47b5-8c5b-7fb0c605cf41 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.807960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959b045-45ad-47b5-8c5b-7fb0c605cf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.808504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c959b045-45ad-47b5-8c5b-7fb0c605cf41 admin admin] PATH_INFO: `/v3/projects/fc5035b7de0d4b9db0d1e9450b855300/users/1740eb931e424df3b31db5d15b11b8b7/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.843770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959b045-45ad-47b5-8c5b-7fb0c605cf41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc5035b7de0d4b9db0d1e9450b855300, user_id=1740eb931e424df3b31db5d15b11b8b7, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:03.847537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c959b045-45ad-47b5-8c5b-7fb0c605cf41 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:03.872944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 988/2001] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:03 2022] PUT /identity/v3/projects/fc5035b7de0d4b9db0d1e9450b855300/users/1740eb931e424df3b31db5d15b11b8b7/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:03.883209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a7fa2-190d-4864-ad4b-c43332afedef None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:03.883545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a7fa2-190d-4864-ad4b-c43332afedef None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:03.883827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1a7fa2-190d-4864-ad4b-c43332afedef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:03.935480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8e1a7fa2-190d-4864-ad4b-c43332afedef None None] MFA Rules not processed for user `1740eb931e424df3b31db5d15b11b8b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:03.936719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8e1a7fa2-190d-4864-ad4b-c43332afedef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:04.026879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1014/2002] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:03 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:04.035346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959b045-45ad-47b5-8c5b-7fb0c605cf41 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.051744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a7fa2-190d-4864-ad4b-c43332afedef None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.053238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8e1a7fa2-190d-4864-ad4b-c43332afedef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:04.120326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c959b045-45ad-47b5-8c5b-7fb0c605cf41 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.125029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a413c3d-c772-4298-8140-1c00ce8506b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.125662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a413c3d-c772-4298-8140-1c00ce8506b7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.125942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a413c3d-c772-4298-8140-1c00ce8506b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.126262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a413c3d-c772-4298-8140-1c00ce8506b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.126906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a413c3d-c772-4298-8140-1c00ce8506b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.128821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a413c3d-c772-4298-8140-1c00ce8506b7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.142681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1a7fa2-190d-4864-ad4b-c43332afedef None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.149569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.150884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.151163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.151437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] PATH_INFO: `/v3/projects/58d0385926c54d0b8d21fc98cfaa21cf` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.152896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58d0385926c54d0b8d21fc98cfaa21cf)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.155887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.160380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 989/2003] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:04 2022] POST /identity/v3/projects => generated 379 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:04.161305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a413c3d-c772-4298-8140-1c00ce8506b7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.164588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9a413c3d-c772-4298-8140-1c00ce8506b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:04.211088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58d0385926c54d0b8d21fc98cfaa21cf', 'request_id': 'req-559e07e3-1740-4b9c-a7d7-f23a900df345'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.218375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58d0385926c54d0b8d21fc98cfaa21cf', 'request_id': 'req-559e07e3-1740-4b9c-a7d7-f23a900df345'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.226236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58d0385926c54d0b8d21fc98cfaa21cf', 'request_id': 'req-559e07e3-1740-4b9c-a7d7-f23a900df345'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.242970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a413c3d-c772-4298-8140-1c00ce8506b7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.248473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] The token cache is being invalidate because project 58d0385926c54d0b8d21fc98cfaa21cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:04.248772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.249442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.251411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1015/2004] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:04 2022] DELETE /identity/v3/projects/58d0385926c54d0b8d21fc98cfaa21cf => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:04.251944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.253800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:04.270535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:04.326060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.326866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.327146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.327453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] PATH_INFO: `/v3/projects/44b2db2ba59f4725b4319ef2cddb5f66` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.328574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b2db2ba59f4725b4319ef2cddb5f66)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.330534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.335106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559e07e3-1740-4b9c-a7d7-f23a900df345 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.340270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe81c2-bee1-41c0-ada0-9007f9412341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.340893 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe81c2-bee1-41c0-ada0-9007f9412341 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.341173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe81c2-bee1-41c0-ada0-9007f9412341 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.341445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fe81c2-bee1-41c0-ada0-9007f9412341 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.342098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe81c2-bee1-41c0-ada0-9007f9412341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.345237 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fe81c2-bee1-41c0-ada0-9007f9412341 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.346823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:04.371420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1016/2005] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:04 2022] POST /identity/v3/users => generated 296 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:04.372301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe81c2-bee1-41c0-ada0-9007f9412341 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.373913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-89fe81c2-bee1-41c0-ada0-9007f9412341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:04.386243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fe81c2-bee1-41c0-ada0-9007f9412341 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.387436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b2db2ba59f4725b4319ef2cddb5f66', 'request_id': 'req-224ada52-6782-4728-9abd-406b3b4e27af'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.393704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b2db2ba59f4725b4319ef2cddb5f66', 'request_id': 'req-224ada52-6782-4728-9abd-406b3b4e27af'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.400433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b2db2ba59f4725b4319ef2cddb5f66', 'request_id': 'req-224ada52-6782-4728-9abd-406b3b4e27af'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.415063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] The token cache is being invalidate because project 44b2db2ba59f4725b4319ef2cddb5f66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:04.415370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.416029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.417540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 990/2006] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:04 2022] DELETE /identity/v3/projects/44b2db2ba59f4725b4319ef2cddb5f66 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:04.421252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.422573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:04.446839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.447499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.447778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.448057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] PATH_INFO: `/v3/users/c8b432a53be2419997acacfc9a7d071d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.449199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8b432a53be2419997acacfc9a7d071d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.451942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.493860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ada52-6782-4728-9abd-406b3b4e27af admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.500276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d259c-d9c2-4703-9cfd-67f7fa7d3219 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.501080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d259c-d9c2-4703-9cfd-67f7fa7d3219 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.501363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d259c-d9c2-4703-9cfd-67f7fa7d3219 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.501627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21d259c-d9c2-4703-9cfd-67f7fa7d3219 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.502336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d259c-d9c2-4703-9cfd-67f7fa7d3219 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.504577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d21d259c-d9c2-4703-9cfd-67f7fa7d3219 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.505226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8b432a53be2419997acacfc9a7d071d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.505951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8b432a53be2419997acacfc9a7d071d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.518500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 991/2007] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:04.518500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d259c-d9c2-4703-9cfd-67f7fa7d3219 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.518757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8b432a53be2419997acacfc9a7d071d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.518836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d21d259c-d9c2-4703-9cfd-67f7fa7d3219 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:04.532112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d21d259c-d9c2-4703-9cfd-67f7fa7d3219 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.535086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8b432a53be2419997acacfc9a7d071d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.535964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8b432a53be2419997acacfc9a7d071d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.537982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.538596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.538878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.539151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] PATH_INFO: `/v3/users/3412679ed4124c36a33ac05c53a33a23` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.540355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3412679ed4124c36a33ac05c53a33a23)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.542536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.550749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1017/2008] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:04 2022] DELETE /identity/v3/users/c8b432a53be2419997acacfc9a7d071d => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:04.551658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.553736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:04.581338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:04.627610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3412679ed4124c36a33ac05c53a33a23'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.640820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3412679ed4124c36a33ac05c53a33a23'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.647754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a3a84a-1b4d-4f93-8fa4-b438c96aa443 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.653544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2400ce-de73-4585-823b-ace3a9b11532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.654870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2400ce-de73-4585-823b-ace3a9b11532 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.655152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2400ce-de73-4585-823b-ace3a9b11532 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.655442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2400ce-de73-4585-823b-ace3a9b11532 admin admin] PATH_INFO: `/v3/projects/cbf39b4bd399494590b43e4de67c3dca/users/4cc5f5d6d2224abd81b1846b7f7ddf30/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.661099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3412679ed4124c36a33ac05c53a33a23'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.661988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3412679ed4124c36a33ac05c53a33a23'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.670611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3412679ed4124c36a33ac05c53a33a23'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:04.673143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2400ce-de73-4585-823b-ace3a9b11532 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf39b4bd399494590b43e4de67c3dca, user_id=4cc5f5d6d2224abd81b1846b7f7ddf30, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.675978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb2400ce-de73-4585-823b-ace3a9b11532 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.678894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 992/2009] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:04 2022] DELETE /identity/v3/users/3412679ed4124c36a33ac05c53a33a23 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:04.692647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1018/2010] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:04 2022] PUT /identity/v3/projects/cbf39b4bd399494590b43e4de67c3dca/users/4cc5f5d6d2224abd81b1846b7f7ddf30/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:04.697233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.699726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:04.712071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:04.784169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89d4850-1c1e-4ba6-ab9e-e9cb46459849 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.844673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8ddf07-bd87-4d24-92b8-7b0fc59283ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.845405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ddf07-bd87-4d24-92b8-7b0fc59283ab admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.845761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ddf07-bd87-4d24-92b8-7b0fc59283ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.846133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8ddf07-bd87-4d24-92b8-7b0fc59283ab admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.846797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ddf07-bd87-4d24-92b8-7b0fc59283ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.848512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8ddf07-bd87-4d24-92b8-7b0fc59283ab admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.862628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 993/2011] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:04 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:04.864813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2400ce-de73-4585-823b-ace3a9b11532 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:04.884254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2400ce-de73-4585-823b-ace3a9b11532 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:04.907146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652692cc-e323-4913-9299-7e9a103256bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:04.910816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652692cc-e323-4913-9299-7e9a103256bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.911228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652692cc-e323-4913-9299-7e9a103256bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.911468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652692cc-e323-4913-9299-7e9a103256bd admin admin] PATH_INFO: `/v3/projects/cbf39b4bd399494590b43e4de67c3dca/users/4cc5f5d6d2224abd81b1846b7f7ddf30/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:04.916255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652692cc-e323-4913-9299-7e9a103256bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbf39b4bd399494590b43e4de67c3dca, user_id=4cc5f5d6d2224abd81b1846b7f7ddf30, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:04.939972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652692cc-e323-4913-9299-7e9a103256bd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:04.959780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1019/2012] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:04 2022] PUT /identity/v3/projects/cbf39b4bd399494590b43e4de67c3dca/users/4cc5f5d6d2224abd81b1846b7f7ddf30/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:04.973230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4e0a3-75f5-4938-bbfe-deafb6847576 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:04.973551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4e0a3-75f5-4938-bbfe-deafb6847576 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:04.973793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f4e0a3-75f5-4938-bbfe-deafb6847576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.006966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-00f4e0a3-75f5-4938-bbfe-deafb6847576 None None] MFA Rules not processed for user `4cc5f5d6d2224abd81b1846b7f7ddf30`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:05.008226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-00f4e0a3-75f5-4938-bbfe-deafb6847576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:05.086198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 994/2013] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:04 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:05.094007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652692cc-e323-4913-9299-7e9a103256bd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:05.095791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-652692cc-e323-4913-9299-7e9a103256bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:05.166537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652692cc-e323-4913-9299-7e9a103256bd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:05.171925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403600a-dd52-47c7-981b-8ad887c07dbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:05.172722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403600a-dd52-47c7-981b-8ad887c07dbb service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.173003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403600a-dd52-47c7-981b-8ad887c07dbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.173281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a403600a-dd52-47c7-981b-8ad887c07dbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.187107 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403600a-dd52-47c7-981b-8ad887c07dbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:05.188855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a403600a-dd52-47c7-981b-8ad887c07dbb service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:05.204359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1020/2014] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:05 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:05.256174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4e0a3-75f5-4938-bbfe-deafb6847576 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:05.258289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-00f4e0a3-75f5-4938-bbfe-deafb6847576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:05.322223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:05.328522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403600a-dd52-47c7-981b-8ad887c07dbb service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:05.330115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a403600a-dd52-47c7-981b-8ad887c07dbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:05.372892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f4e0a3-75f5-4938-bbfe-deafb6847576 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:05.378432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:05.379223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.379551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.379956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] PATH_INFO: `/v3/projects/6d3da5f651a54857a1da99125487d3ef` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.381492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d3da5f651a54857a1da99125487d3ef)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:05.384890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:05.449041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a403600a-dd52-47c7-981b-8ad887c07dbb service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:05.454732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:05.455571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.455976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.456372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] PATH_INFO: `/v3/projects/5c02a16e2b1c4c2fa04075f182d44bac` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.457945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c02a16e2b1c4c2fa04075f182d44bac)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:05.459898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:05.483196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d3da5f651a54857a1da99125487d3ef', 'request_id': 'req-615c0e8e-1a40-4ef6-9b13-2388776ca18e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.490935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d3da5f651a54857a1da99125487d3ef', 'request_id': 'req-615c0e8e-1a40-4ef6-9b13-2388776ca18e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.497333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d3da5f651a54857a1da99125487d3ef', 'request_id': 'req-615c0e8e-1a40-4ef6-9b13-2388776ca18e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.521391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] The token cache is being invalidate because project 6d3da5f651a54857a1da99125487d3ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:05.522032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.523208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-615c0e8e-1a40-4ef6-9b13-2388776ca18e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.528064 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 995/2015] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:05 2022] DELETE /identity/v3/projects/6d3da5f651a54857a1da99125487d3ef => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:05.533606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767c488-e874-43f6-a2a0-7522d1d9f4ec None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.533898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767c488-e874-43f6-a2a0-7522d1d9f4ec None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.534211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767c488-e874-43f6-a2a0-7522d1d9f4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.536070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:05.558795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c02a16e2b1c4c2fa04075f182d44bac', 'request_id': 'req-2f23c460-80ed-4b4b-8a77-ff29817a5d94'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.566055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0767c488-e874-43f6-a2a0-7522d1d9f4ec None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:05.568808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0767c488-e874-43f6-a2a0-7522d1d9f4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:05.569672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c02a16e2b1c4c2fa04075f182d44bac', 'request_id': 'req-2f23c460-80ed-4b4b-8a77-ff29817a5d94'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.578262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c02a16e2b1c4c2fa04075f182d44bac', 'request_id': 'req-2f23c460-80ed-4b4b-8a77-ff29817a5d94'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.609126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] The token cache is being invalidate because project 5c02a16e2b1c4c2fa04075f182d44bac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:05.609299 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.625965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f23c460-80ed-4b4b-8a77-ff29817a5d94 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:05.632837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1021/2016] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:05 2022] DELETE /identity/v3/projects/5c02a16e2b1c4c2fa04075f182d44bac => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:05.634404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df4166-3544-4d87-bf24-ead4d836340c None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.634688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df4166-3544-4d87-bf24-ead4d836340c None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.634958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5df4166-3544-4d87-bf24-ead4d836340c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.663037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c5df4166-3544-4d87-bf24-ead4d836340c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:05.667119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 996/2017] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:05.668610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406b2fe-89e4-430c-98b1-a32b698daa24 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.668906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406b2fe-89e4-430c-98b1-a32b698daa24 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.669186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7406b2fe-89e4-430c-98b1-a32b698daa24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.678915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c5df4166-3544-4d87-bf24-ead4d836340c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:05.692434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7406b2fe-89e4-430c-98b1-a32b698daa24 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:05.693819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7406b2fe-89e4-430c-98b1-a32b698daa24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:05.701965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1022/2018] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:05.709873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 997/2019] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:05.710389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df4166-3544-4d87-bf24-ead4d836340c None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:05.713383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653557a1-d7b2-4b66-9159-a6025c2a96dc None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.714355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653557a1-d7b2-4b66-9159-a6025c2a96dc None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.715528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653557a1-d7b2-4b66-9159-a6025c2a96dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.724029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5df4166-3544-4d87-bf24-ead4d836340c None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:05.730473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c536f3a-8c36-4408-9d1e-ddb66705e2e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:05.731362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c536f3a-8c36-4408-9d1e-ddb66705e2e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.731599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c536f3a-8c36-4408-9d1e-ddb66705e2e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.731993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c536f3a-8c36-4408-9d1e-ddb66705e2e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.732894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c536f3a-8c36-4408-9d1e-ddb66705e2e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:05.733298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c536f3a-8c36-4408-9d1e-ddb66705e2e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:05.735116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c536f3a-8c36-4408-9d1e-ddb66705e2e4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:05.747997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-653557a1-d7b2-4b66-9159-a6025c2a96dc None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:05.749942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-653557a1-d7b2-4b66-9159-a6025c2a96dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:05.761297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1023/2020] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:02:05.762351 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c536f3a-8c36-4408-9d1e-ddb66705e2e4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:05.769997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 998/2021] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:05.775114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653557a1-d7b2-4b66-9159-a6025c2a96dc None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:05.780394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c536f3a-8c36-4408-9d1e-ddb66705e2e4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:05.786016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653557a1-d7b2-4b66-9159-a6025c2a96dc None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:05.789903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83c13d-112c-4c71-81ac-9c053af33867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:05.791485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83c13d-112c-4c71-81ac-9c053af33867 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.791879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83c13d-112c-4c71-81ac-9c053af33867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.792256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a83c13d-112c-4c71-81ac-9c053af33867 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.792604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:05.793108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83c13d-112c-4c71-81ac-9c053af33867 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:05.793704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.794314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.794734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.795597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:05.796042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:05.797088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a83c13d-112c-4c71-81ac-9c053af33867 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:05.798924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:05.828972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 999/2022] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:02:05.832310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:05.834870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:05.838248 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1024/2023] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:05 2022] POST /identity/v3/projects => generated 409 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:05.839053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83c13d-112c-4c71-81ac-9c053af33867 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:05.852314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeed225-05cf-47e7-8781-554d2553ee7a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:05.901409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:05.912708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a83c13d-112c-4c71-81ac-9c053af33867 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:05.922330 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9121d-cfb1-41d7-af81-e13755dba8b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:05.922966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9121d-cfb1-41d7-af81-e13755dba8b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.923245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9121d-cfb1-41d7-af81-e13755dba8b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.923515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa9121d-cfb1-41d7-af81-e13755dba8b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.924160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9121d-cfb1-41d7-af81-e13755dba8b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:05.926407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa9121d-cfb1-41d7-af81-e13755dba8b0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:05.937215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:05.937882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:05.939812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:05.940103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] PATH_INFO: `/v3/users/1740eb931e424df3b31db5d15b11b8b7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:05.942808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1740eb931e424df3b31db5d15b11b8b7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:05.944932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:05.965836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1025/2024] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:05 2022] POST /identity/v3/projects => generated 375 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:05.967592 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9121d-cfb1-41d7-af81-e13755dba8b0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.008279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1740eb931e424df3b31db5d15b11b8b7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.019691 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1740eb931e424df3b31db5d15b11b8b7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.034840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1740eb931e424df3b31db5d15b11b8b7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.036467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1740eb931e424df3b31db5d15b11b8b7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.045213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1740eb931e424df3b31db5d15b11b8b7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.051937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1000/2025] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:05 2022] DELETE /identity/v3/users/1740eb931e424df3b31db5d15b11b8b7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:06.053018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.057032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:06.074279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa9121d-cfb1-41d7-af81-e13755dba8b0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.092874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:06.140836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049e3bda-7161-4a4a-99ef-14eaf4b9fc5a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.146007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d0c95-3ce2-4a57-9c70-0ff7926a8673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.146826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d0c95-3ce2-4a57-9c70-0ff7926a8673 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.147202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d0c95-3ce2-4a57-9c70-0ff7926a8673 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.147549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d0c95-3ce2-4a57-9c70-0ff7926a8673 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.148269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d0c95-3ce2-4a57-9c70-0ff7926a8673 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.150646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d0c95-3ce2-4a57-9c70-0ff7926a8673 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.175647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f2d80-a1bf-40f8-a737-fb3f661edeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.176769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f2d80-a1bf-40f8-a737-fb3f661edeb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.177245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f2d80-a1bf-40f8-a737-fb3f661edeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.177752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-584f2d80-a1bf-40f8-a737-fb3f661edeb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.178677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f2d80-a1bf-40f8-a737-fb3f661edeb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.180406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1001/2026] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:06 2022] POST /identity/v3/users => generated 288 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:06.181823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-584f2d80-a1bf-40f8-a737-fb3f661edeb6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.183755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f963d6-5cdc-4755-ae26-edfe8cd3e896 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.184704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f963d6-5cdc-4755-ae26-edfe8cd3e896 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.185080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f963d6-5cdc-4755-ae26-edfe8cd3e896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.210025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1026/2027] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:05 2022] POST /identity/v3/users => generated 305 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:06.211576 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f2d80-a1bf-40f8-a737-fb3f661edeb6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.215584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-03f963d6-5cdc-4755-ae26-edfe8cd3e896 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:06.216805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-03f963d6-5cdc-4755-ae26-edfe8cd3e896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:06.223753 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-584f2d80-a1bf-40f8-a737-fb3f661edeb6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.224985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1002/2028] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:06.225814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f963d6-5cdc-4755-ae26-edfe8cd3e896 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.227217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-03f963d6-5cdc-4755-ae26-edfe8cd3e896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:06.234925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24394b9b-f2ae-4594-8372-c21f6929401b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.235858 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24394b9b-f2ae-4594-8372-c21f6929401b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.236148 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24394b9b-f2ae-4594-8372-c21f6929401b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.236431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24394b9b-f2ae-4594-8372-c21f6929401b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.237128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24394b9b-f2ae-4594-8372-c21f6929401b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.239732 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f963d6-5cdc-4755-ae26-edfe8cd3e896 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.245772 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330b788-a29c-4cfd-b590-ec6a9f75365f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.246343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24394b9b-f2ae-4594-8372-c21f6929401b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.246698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330b788-a29c-4cfd-b590-ec6a9f75365f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.247111 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330b788-a29c-4cfd-b590-ec6a9f75365f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.247524 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d330b788-a29c-4cfd-b590-ec6a9f75365f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.248474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330b788-a29c-4cfd-b590-ec6a9f75365f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.250607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d330b788-a29c-4cfd-b590-ec6a9f75365f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.268049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1003/2029] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:06.268049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330b788-a29c-4cfd-b590-ec6a9f75365f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.268616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d330b788-a29c-4cfd-b590-ec6a9f75365f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:06.273676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1027/2030] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:06.274139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24394b9b-f2ae-4594-8372-c21f6929401b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.287966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24394b9b-f2ae-4594-8372-c21f6929401b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.296686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26472c9-5a69-4aa7-be26-0c300e6decc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.297888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26472c9-5a69-4aa7-be26-0c300e6decc6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.298273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26472c9-5a69-4aa7-be26-0c300e6decc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.298559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26472c9-5a69-4aa7-be26-0c300e6decc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.301497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26472c9-5a69-4aa7-be26-0c300e6decc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.306331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26472c9-5a69-4aa7-be26-0c300e6decc6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.320169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1028/2031] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:06.320943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26472c9-5a69-4aa7-be26-0c300e6decc6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.327900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:06.336879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26472c9-5a69-4aa7-be26-0c300e6decc6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.342439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4e711-7e66-4346-a69f-81450908cb63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.343206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4e711-7e66-4346-a69f-81450908cb63 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.343557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4e711-7e66-4346-a69f-81450908cb63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.343985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c4e711-7e66-4346-a69f-81450908cb63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.344687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4e711-7e66-4346-a69f-81450908cb63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.346622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c4e711-7e66-4346-a69f-81450908cb63 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.361025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1029/2032] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:06.361025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4e711-7e66-4346-a69f-81450908cb63 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.364153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d330b788-a29c-4cfd-b590-ec6a9f75365f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.368371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7079927-bfe0-47c6-ba89-2ed4bcaff8f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.369005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7079927-bfe0-47c6-ba89-2ed4bcaff8f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.369319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7079927-bfe0-47c6-ba89-2ed4bcaff8f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.369593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7079927-bfe0-47c6-ba89-2ed4bcaff8f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.371565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c4e711-7e66-4346-a69f-81450908cb63 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.376422 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc15dbc-4ba1-475b-8dd1-fa2c0d415cff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.377165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc15dbc-4ba1-475b-8dd1-fa2c0d415cff admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.377449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc15dbc-4ba1-475b-8dd1-fa2c0d415cff admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.377727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc15dbc-4ba1-475b-8dd1-fa2c0d415cff admin admin] PATH_INFO: `/v3/projects/9863686340f4420c85b483f7ae31a747/users/9b55115ba4594da9bb14a1731312113a/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.381417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7079927-bfe0-47c6-ba89-2ed4bcaff8f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.383100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7079927-bfe0-47c6-ba89-2ed4bcaff8f1 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.393715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1004/2033] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:06.394405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7079927-bfe0-47c6-ba89-2ed4bcaff8f1 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.401527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc15dbc-4ba1-475b-8dd1-fa2c0d415cff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9863686340f4420c85b483f7ae31a747, user_id=9b55115ba4594da9bb14a1731312113a, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.404839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7079927-bfe0-47c6-ba89-2ed4bcaff8f1 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.406224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc15dbc-4ba1-475b-8dd1-fa2c0d415cff admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.410578 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90715f2-8272-4d57-8400-d7ddf023fc9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.411375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90715f2-8272-4d57-8400-d7ddf023fc9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.411729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90715f2-8272-4d57-8400-d7ddf023fc9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.412076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90715f2-8272-4d57-8400-d7ddf023fc9a admin admin] PATH_INFO: `/v3/projects/f69e7bcc7db34476b689772070c2f726/users/7803cc1f848442f9be04ba265d444530/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.429114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1030/2034] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:06 2022] PUT /identity/v3/projects/9863686340f4420c85b483f7ae31a747/users/9b55115ba4594da9bb14a1731312113a/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:06.430241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90715f2-8272-4d57-8400-d7ddf023fc9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f69e7bcc7db34476b689772070c2f726, user_id=7803cc1f848442f9be04ba265d444530, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.433567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90715f2-8272-4d57-8400-d7ddf023fc9a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.443721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed02a4-9e2a-4e52-9832-f28222cec13f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.444498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed02a4-9e2a-4e52-9832-f28222cec13f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.444982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eed02a4-9e2a-4e52-9832-f28222cec13f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.452196 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1005/2035] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:06 2022] PUT /identity/v3/projects/f69e7bcc7db34476b689772070c2f726/users/7803cc1f848442f9be04ba265d444530/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:06.464006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72633ce7-4f23-4c8b-b4bb-d41cbeb03f7a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.464432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72633ce7-4f23-4c8b-b4bb-d41cbeb03f7a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.464927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72633ce7-4f23-4c8b-b4bb-d41cbeb03f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.481637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7eed02a4-9e2a-4e52-9832-f28222cec13f None None] MFA Rules not processed for user `9b55115ba4594da9bb14a1731312113a`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:06.483495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7eed02a4-9e2a-4e52-9832-f28222cec13f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:06.509233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-72633ce7-4f23-4c8b-b4bb-d41cbeb03f7a None None] MFA Rules not processed for user `7803cc1f848442f9be04ba265d444530`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:06.510815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-72633ce7-4f23-4c8b-b4bb-d41cbeb03f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:06.603040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1031/2036] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:06 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:06.608002 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed02a4-9e2a-4e52-9832-f28222cec13f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.636040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1006/2037] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:06.640876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72633ce7-4f23-4c8b-b4bb-d41cbeb03f7a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.654297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:06.681733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eed02a4-9e2a-4e52-9832-f28222cec13f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.687561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455103c-bacb-4af5-8257-73deb8d01719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.688269 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455103c-bacb-4af5-8257-73deb8d01719 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.688647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455103c-bacb-4af5-8257-73deb8d01719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.688928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f455103c-bacb-4af5-8257-73deb8d01719 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.689571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455103c-bacb-4af5-8257-73deb8d01719 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.691682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f455103c-bacb-4af5-8257-73deb8d01719 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.712507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72633ce7-4f23-4c8b-b4bb-d41cbeb03f7a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.717429 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fdf05-1bbf-47f2-bbce-d9f51fdda62b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.718057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fdf05-1bbf-47f2-bbce-d9f51fdda62b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.718553 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fdf05-1bbf-47f2-bbce-d9f51fdda62b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.718830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469fdf05-1bbf-47f2-bbce-d9f51fdda62b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.719470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fdf05-1bbf-47f2-bbce-d9f51fdda62b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.721497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469fdf05-1bbf-47f2-bbce-d9f51fdda62b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.743879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1032/2038] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:06 2022] POST /identity/v3/projects => generated 409 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:06.744775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455103c-bacb-4af5-8257-73deb8d01719 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.746703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f455103c-bacb-4af5-8257-73deb8d01719 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:06.758367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1007/2039] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:06 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:06.759363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fdf05-1bbf-47f2-bbce-d9f51fdda62b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.795393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:06.827835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f455103c-bacb-4af5-8257-73deb8d01719 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.841063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469fdf05-1bbf-47f2-bbce-d9f51fdda62b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.845910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf315bb-288f-4d1c-9d7e-441197f0be64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.846676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf315bb-288f-4d1c-9d7e-441197f0be64 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.847045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf315bb-288f-4d1c-9d7e-441197f0be64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.847378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf315bb-288f-4d1c-9d7e-441197f0be64 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.848655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf315bb-288f-4d1c-9d7e-441197f0be64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.850163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf315bb-288f-4d1c-9d7e-441197f0be64 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.882199 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1008/2040] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:06 2022] POST /identity/v3/users => generated 311 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:06.883056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf315bb-288f-4d1c-9d7e-441197f0be64 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.892646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf315bb-288f-4d1c-9d7e-441197f0be64 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.897559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e223369-9d05-4d84-bf7b-25624daf9f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.898199 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e223369-9d05-4d84-bf7b-25624daf9f36 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.898484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e223369-9d05-4d84-bf7b-25624daf9f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.898752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e223369-9d05-4d84-bf7b-25624daf9f36 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.899377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e223369-9d05-4d84-bf7b-25624daf9f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.901287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e223369-9d05-4d84-bf7b-25624daf9f36 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.904807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.905392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.905676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.905952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] PATH_INFO: `/v3/projects/fc5035b7de0d4b9db0d1e9450b855300` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.907130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc5035b7de0d4b9db0d1e9450b855300)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.909085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.928992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1009/2041] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:06 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:06.928992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e223369-9d05-4d84-bf7b-25624daf9f36 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.937320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e223369-9d05-4d84-bf7b-25624daf9f36 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.942042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aabbaf-3b94-4711-b071-f67808bee099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:06.942679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aabbaf-3b94-4711-b071-f67808bee099 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:06.942952 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aabbaf-3b94-4711-b071-f67808bee099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:06.943228 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aabbaf-3b94-4711-b071-f67808bee099 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:06.943902 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aabbaf-3b94-4711-b071-f67808bee099 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:06.945498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52aabbaf-3b94-4711-b071-f67808bee099 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:06.956095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1010/2042] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:06.956970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aabbaf-3b94-4711-b071-f67808bee099 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.959360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc5035b7de0d4b9db0d1e9450b855300', 'request_id': 'req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.966359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc5035b7de0d4b9db0d1e9450b855300', 'request_id': 'req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.966969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aabbaf-3b94-4711-b071-f67808bee099 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:06.972348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc5035b7de0d4b9db0d1e9450b855300', 'request_id': 'req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.981545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:06.986853 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] The token cache is being invalidate because project fc5035b7de0d4b9db0d1e9450b855300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:06.987473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.988749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:06.990689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1033/2043] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:06 2022] DELETE /identity/v3/projects/fc5035b7de0d4b9db0d1e9450b855300 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:06.991884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:06.994603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.034893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cde6f9-d723-4050-baec-b9a44688d7af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.035514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cde6f9-d723-4050-baec-b9a44688d7af admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.035943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cde6f9-d723-4050-baec-b9a44688d7af admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.036237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cde6f9-d723-4050-baec-b9a44688d7af admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.037005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cde6f9-d723-4050-baec-b9a44688d7af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.038691 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cde6f9-d723-4050-baec-b9a44688d7af admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.049032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1011/2044] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:06 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:07.049957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cde6f9-d723-4050-baec-b9a44688d7af admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.053162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a4cde6f9-d723-4050-baec-b9a44688d7af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.065811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cde6f9-d723-4050-baec-b9a44688d7af admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.069198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ebf5f7-8a8f-4ec5-977a-96712b7bae54 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.070728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.071931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.072225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.072502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] PATH_INFO: `/v3/users/4cc5f5d6d2224abd81b1846b7f7ddf30` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.076189 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc5f5d6d2224abd81b1846b7f7ddf30)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.076296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea7326-f537-4981-a038-ee4538d20438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.076296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea7326-f537-4981-a038-ee4538d20438 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.076296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea7326-f537-4981-a038-ee4538d20438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.076296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ea7326-f537-4981-a038-ee4538d20438 admin admin] PATH_INFO: `/v3/projects/747bd511d8124fa7bc986479168af4b0/users/ac8df33c65b94319b455c89d7a55c2e8/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.076572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.090579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea7326-f537-4981-a038-ee4538d20438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747bd511d8124fa7bc986479168af4b0, user_id=ac8df33c65b94319b455c89d7a55c2e8, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.093259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ea7326-f537-4981-a038-ee4538d20438 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.108226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1034/2045] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:06 2022] PUT /identity/v3/projects/747bd511d8124fa7bc986479168af4b0/users/ac8df33c65b94319b455c89d7a55c2e8/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:07.109106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea7326-f537-4981-a038-ee4538d20438 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.110699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c8ea7326-f537-4981-a038-ee4538d20438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.128179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc5f5d6d2224abd81b1846b7f7ddf30'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.137424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc5f5d6d2224abd81b1846b7f7ddf30'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.149802 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc5f5d6d2224abd81b1846b7f7ddf30'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.150500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc5f5d6d2224abd81b1846b7f7ddf30'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.155100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83,keystone:INSERT=7,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:07.159854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc5f5d6d2224abd81b1846b7f7ddf30'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.166997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1012/2046] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:07 2022] DELETE /identity/v3/users/4cc5f5d6d2224abd81b1846b7f7ddf30 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:07.167672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.169048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.177117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ea7326-f537-4981-a038-ee4538d20438 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.243204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0385f4-d827-4ebb-98cd-b7571f9b5736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.243874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0385f4-d827-4ebb-98cd-b7571f9b5736 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.244156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0385f4-d827-4ebb-98cd-b7571f9b5736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.244514 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0385f4-d827-4ebb-98cd-b7571f9b5736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.245222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0385f4-d827-4ebb-98cd-b7571f9b5736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.247046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0385f4-d827-4ebb-98cd-b7571f9b5736 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.247514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3306f13e-88e1-4f31-82d3-e9d12584868c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.253833 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d27ac0-1d43-4d1f-aebf-01d046537cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.254577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d27ac0-1d43-4d1f-aebf-01d046537cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.254946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d27ac0-1d43-4d1f-aebf-01d046537cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.255275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d27ac0-1d43-4d1f-aebf-01d046537cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.255952 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d27ac0-1d43-4d1f-aebf-01d046537cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.257559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d27ac0-1d43-4d1f-aebf-01d046537cfc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.258424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1035/2047] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:07.262656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0385f4-d827-4ebb-98cd-b7571f9b5736 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.264519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dc0385f4-d827-4ebb-98cd-b7571f9b5736 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.271816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1013/2048] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:07.275889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d27ac0-1d43-4d1f-aebf-01d046537cfc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.277543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0385f4-d827-4ebb-98cd-b7571f9b5736 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.283825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c659326-9d10-4304-87b1-90eccd5228e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.284499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659326-9d10-4304-87b1-90eccd5228e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.284801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659326-9d10-4304-87b1-90eccd5228e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.285074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c659326-9d10-4304-87b1-90eccd5228e3 admin admin] PATH_INFO: `/v3/projects/0bd8d04167d64e99891a4d01faf7f518/users/33abb44c09424991a1fc169260377662/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.290907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d27ac0-1d43-4d1f-aebf-01d046537cfc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.296115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d23bcff-f646-4f7d-94a5-56a15b7ff163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.296811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23bcff-f646-4f7d-94a5-56a15b7ff163 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.297180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23bcff-f646-4f7d-94a5-56a15b7ff163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.297499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d23bcff-f646-4f7d-94a5-56a15b7ff163 admin admin] PATH_INFO: `/v3/projects/747bd511d8124fa7bc986479168af4b0/users/ac8df33c65b94319b455c89d7a55c2e8/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.299660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23bcff-f646-4f7d-94a5-56a15b7ff163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=747bd511d8124fa7bc986479168af4b0, user_id=ac8df33c65b94319b455c89d7a55c2e8, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.302511 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d23bcff-f646-4f7d-94a5-56a15b7ff163 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.302685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c659326-9d10-4304-87b1-90eccd5228e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0bd8d04167d64e99891a4d01faf7f518, user_id=33abb44c09424991a1fc169260377662, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.305490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c659326-9d10-4304-87b1-90eccd5228e3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.314904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1014/2049] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:07 2022] PUT /identity/v3/projects/747bd511d8124fa7bc986479168af4b0/users/ac8df33c65b94319b455c89d7a55c2e8/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:07.326477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:07.327707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e782db-2cc6-4f41-9d6a-aca776286742 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.328068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e782db-2cc6-4f41-9d6a-aca776286742 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.328503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e782db-2cc6-4f41-9d6a-aca776286742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.340905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1036/2050] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:07 2022] PUT /identity/v3/projects/0bd8d04167d64e99891a4d01faf7f518/users/33abb44c09424991a1fc169260377662/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:07.344755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2278b53-73d0-4507-8af1-7da7f458eb33 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.345098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2278b53-73d0-4507-8af1-7da7f458eb33 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.345434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2278b53-73d0-4507-8af1-7da7f458eb33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.359526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-48e782db-2cc6-4f41-9d6a-aca776286742 None None] MFA Rules not processed for user `ac8df33c65b94319b455c89d7a55c2e8`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:07.360744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-48e782db-2cc6-4f41-9d6a-aca776286742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.380178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d2278b53-73d0-4507-8af1-7da7f458eb33 None None] MFA Rules not processed for user `33abb44c09424991a1fc169260377662`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:07.383043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d2278b53-73d0-4507-8af1-7da7f458eb33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.444644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1015/2051] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:07 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:07.452178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e782db-2cc6-4f41-9d6a-aca776286742 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.453565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-48e782db-2cc6-4f41-9d6a-aca776286742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.460919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1037/2052] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:07.465423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2278b53-73d0-4507-8af1-7da7f458eb33 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.521531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e782db-2cc6-4f41-9d6a-aca776286742 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.525295 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085a1da-a258-40fd-a115-7fa3264f2125 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.526000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085a1da-a258-40fd-a115-7fa3264f2125 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.526367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085a1da-a258-40fd-a115-7fa3264f2125 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.526794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3085a1da-a258-40fd-a115-7fa3264f2125 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.538850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2278b53-73d0-4507-8af1-7da7f458eb33 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.543714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d403b07-5e90-4854-9622-fd7ad6fab195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.544430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085a1da-a258-40fd-a115-7fa3264f2125 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.544613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d403b07-5e90-4854-9622-fd7ad6fab195 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.545305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d403b07-5e90-4854-9622-fd7ad6fab195 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.545576 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d403b07-5e90-4854-9622-fd7ad6fab195 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.545962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3085a1da-a258-40fd-a115-7fa3264f2125 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.546405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d403b07-5e90-4854-9622-fd7ad6fab195 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.548487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d403b07-5e90-4854-9622-fd7ad6fab195 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.558416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1016/2053] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:07 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:02:07.565636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085a1da-a258-40fd-a115-7fa3264f2125 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.569419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3085a1da-a258-40fd-a115-7fa3264f2125 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.589311 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1038/2054] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:07 2022] POST /identity/v3/projects => generated 377 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:07.592586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d403b07-5e90-4854-9622-fd7ad6fab195 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.595477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:07.605055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3085a1da-a258-40fd-a115-7fa3264f2125 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.692323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.693143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.693598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.693940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] PATH_INFO: `/v3/projects/cbf39b4bd399494590b43e4de67c3dca` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.695310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbf39b4bd399494590b43e4de67c3dca)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.698117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.710567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d403b07-5e90-4854-9622-fd7ad6fab195 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.716391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5ddc-92af-428b-ae71-0ea5e0dfd5d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.716980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5ddc-92af-428b-ae71-0ea5e0dfd5d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.717323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5ddc-92af-428b-ae71-0ea5e0dfd5d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.717601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62f5ddc-92af-428b-ae71-0ea5e0dfd5d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.718382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f5ddc-92af-428b-ae71-0ea5e0dfd5d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.720593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62f5ddc-92af-428b-ae71-0ea5e0dfd5d3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.754619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1039/2055] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:07 2022] POST /identity/v3/users => generated 295 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:07.759313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5ddc-92af-428b-ae71-0ea5e0dfd5d3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.768054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:07.771698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbf39b4bd399494590b43e4de67c3dca', 'request_id': 'req-29f54227-e272-4149-b9fe-0feacf55c8f3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.772788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62f5ddc-92af-428b-ae71-0ea5e0dfd5d3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.780528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbf39b4bd399494590b43e4de67c3dca', 'request_id': 'req-29f54227-e272-4149-b9fe-0feacf55c8f3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.786887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbf39b4bd399494590b43e4de67c3dca', 'request_id': 'req-29f54227-e272-4149-b9fe-0feacf55c8f3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.805704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] The token cache is being invalidate because project cbf39b4bd399494590b43e4de67c3dca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:07.806363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.807087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:07.809105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1017/2056] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:07 2022] DELETE /identity/v3/projects/cbf39b4bd399494590b43e4de67c3dca => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:07.809900 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.811854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.848442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dcdc64-cef8-42c6-af92-46627503d011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.849064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcdc64-cef8-42c6-af92-46627503d011 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.849345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcdc64-cef8-42c6-af92-46627503d011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.849613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dcdc64-cef8-42c6-af92-46627503d011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.850325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcdc64-cef8-42c6-af92-46627503d011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.852001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dcdc64-cef8-42c6-af92-46627503d011 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.866119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1040/2057] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:07.866119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaa519-8632-40f8-b40d-d3dd42915ef4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.866119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaa519-8632-40f8-b40d-d3dd42915ef4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.866119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ceaa519-8632-40f8-b40d-d3dd42915ef4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.881832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f54227-e272-4149-b9fe-0feacf55c8f3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.884382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2ceaa519-8632-40f8-b40d-d3dd42915ef4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:07.885447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b533ce0-6526-4dfb-9668-136abdc6f1ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.885657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2ceaa519-8632-40f8-b40d-d3dd42915ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.886066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b533ce0-6526-4dfb-9668-136abdc6f1ce service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.886368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b533ce0-6526-4dfb-9668-136abdc6f1ce service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.886752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b533ce0-6526-4dfb-9668-136abdc6f1ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.888448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3b533ce0-6526-4dfb-9668-136abdc6f1ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.893542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1041/2058] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:07.894308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaa519-8632-40f8-b40d-d3dd42915ef4 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.895952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2ceaa519-8632-40f8-b40d-d3dd42915ef4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.921707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ceaa519-8632-40f8-b40d-d3dd42915ef4 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:07.926788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b42561-c449-467e-9446-65744a5743a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:07.927423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b42561-c449-467e-9446-65744a5743a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.927698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b42561-c449-467e-9446-65744a5743a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.927967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b42561-c449-467e-9446-65744a5743a3 admin admin] PATH_INFO: `/v3/projects/a0c95c90475e4f6caa4d5cbf7b33e356/users/a11a5ff1a3d54379baa7a4aa6fe84cb2/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.943427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b42561-c449-467e-9446-65744a5743a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c95c90475e4f6caa4d5cbf7b33e356, user_id=a11a5ff1a3d54379baa7a4aa6fe84cb2, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.946725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b42561-c449-467e-9446-65744a5743a3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.951926 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:07.965974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1042/2059] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:07 2022] PUT /identity/v3/projects/a0c95c90475e4f6caa4d5cbf7b33e356/users/a11a5ff1a3d54379baa7a4aa6fe84cb2/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:07.967453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44cd2e-7cb8-4a1b-b34e-6f636067107b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:07.968043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44cd2e-7cb8-4a1b-b34e-6f636067107b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:07.968173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af44cd2e-7cb8-4a1b-b34e-6f636067107b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:07.970439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b533ce0-6526-4dfb-9668-136abdc6f1ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:07.971982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b533ce0-6526-4dfb-9668-136abdc6f1ce service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:07.982039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1018/2060] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:07 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:07.983340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b533ce0-6526-4dfb-9668-136abdc6f1ce service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:07.992301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-af44cd2e-7cb8-4a1b-b34e-6f636067107b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:07.993755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-af44cd2e-7cb8-4a1b-b34e-6f636067107b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:07.994783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b533ce0-6526-4dfb-9668-136abdc6f1ce service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.069019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1043/2061] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:08.073849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44cd2e-7cb8-4a1b-b34e-6f636067107b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:08.084048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af44cd2e-7cb8-4a1b-b34e-6f636067107b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.101267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bea055-454e-4da1-94df-6bae7cd38dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.101810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bea055-454e-4da1-94df-6bae7cd38dfa admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.102199 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bea055-454e-4da1-94df-6bae7cd38dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.102447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bea055-454e-4da1-94df-6bae7cd38dfa admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.102806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae65628-5cc8-4792-b625-650d7acb2866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.103033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bea055-454e-4da1-94df-6bae7cd38dfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.103558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae65628-5cc8-4792-b625-650d7acb2866 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.103899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae65628-5cc8-4792-b625-650d7acb2866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.104245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae65628-5cc8-4792-b625-650d7acb2866 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.104811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bea055-454e-4da1-94df-6bae7cd38dfa admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.104965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae65628-5cc8-4792-b625-650d7acb2866 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:08.105396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae65628-5cc8-4792-b625-650d7acb2866 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.108100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae65628-5cc8-4792-b625-650d7acb2866 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.119256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1019/2062] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:07 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:08.120169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bea055-454e-4da1-94df-6bae7cd38dfa admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:08.124022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1044/2063] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:08.128170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae65628-5cc8-4792-b625-650d7acb2866 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:08.131164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bea055-454e-4da1-94df-6bae7cd38dfa admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.135943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499011eb-953d-4532-a09d-59fc40650580 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.136627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499011eb-953d-4532-a09d-59fc40650580 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.136906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499011eb-953d-4532-a09d-59fc40650580 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.137177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499011eb-953d-4532-a09d-59fc40650580 admin admin] PATH_INFO: `/v3/projects/a0c95c90475e4f6caa4d5cbf7b33e356/users/a11a5ff1a3d54379baa7a4aa6fe84cb2/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.138411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae65628-5cc8-4792-b625-650d7acb2866 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.139271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499011eb-953d-4532-a09d-59fc40650580 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0c95c90475e4f6caa4d5cbf7b33e356, user_id=a11a5ff1a3d54379baa7a4aa6fe84cb2, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.141932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499011eb-953d-4532-a09d-59fc40650580 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.145487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ce190-53d9-4fbe-b942-134801eb294c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.146300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ce190-53d9-4fbe-b942-134801eb294c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.146633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ce190-53d9-4fbe-b942-134801eb294c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.146864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ce190-53d9-4fbe-b942-134801eb294c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.147601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ce190-53d9-4fbe-b942-134801eb294c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.149489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ce190-53d9-4fbe-b942-134801eb294c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.153755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1020/2064] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:08 2022] PUT /identity/v3/projects/a0c95c90475e4f6caa4d5cbf7b33e356/users/a11a5ff1a3d54379baa7a4aa6fe84cb2/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:08.165389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9ba3-afcc-4578-aae6-f861058af305 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.165733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9ba3-afcc-4578-aae6-f861058af305 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.166108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fddd9ba3-afcc-4578-aae6-f861058af305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.188136 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1045/2065] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:08 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:08.189643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ce190-53d9-4fbe-b942-134801eb294c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:08.200268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fddd9ba3-afcc-4578-aae6-f861058af305 None None] MFA Rules not processed for user `a11a5ff1a3d54379baa7a4aa6fe84cb2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:08.201585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fddd9ba3-afcc-4578-aae6-f861058af305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:08.234011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:08.265470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ce190-53d9-4fbe-b942-134801eb294c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.271296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d470602-8301-4801-b038-d8557c4027cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.272173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d470602-8301-4801-b038-d8557c4027cf admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.273147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d470602-8301-4801-b038-d8557c4027cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.273447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d470602-8301-4801-b038-d8557c4027cf admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.274235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d470602-8301-4801-b038-d8557c4027cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.276187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d470602-8301-4801-b038-d8557c4027cf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.287451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1021/2066] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:08.306488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9ba3-afcc-4578-aae6-f861058af305 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:08.307970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fddd9ba3-afcc-4578-aae6-f861058af305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:08.313175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1046/2067] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:08 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:08.319647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d470602-8301-4801-b038-d8557c4027cf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:08.330972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d470602-8301-4801-b038-d8557c4027cf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.340027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77971d9e-2dfe-4589-84a9-5ba46ab574f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.341131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77971d9e-2dfe-4589-84a9-5ba46ab574f8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.341416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77971d9e-2dfe-4589-84a9-5ba46ab574f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.341923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77971d9e-2dfe-4589-84a9-5ba46ab574f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.342606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77971d9e-2dfe-4589-84a9-5ba46ab574f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.349039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77971d9e-2dfe-4589-84a9-5ba46ab574f8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.383618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1047/2068] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:08.385013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77971d9e-2dfe-4589-84a9-5ba46ab574f8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:08.396814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77971d9e-2dfe-4589-84a9-5ba46ab574f8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.397138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fddd9ba3-afcc-4578-aae6-f861058af305 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.401100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3f9a1-27e5-496f-897b-ab07bb8a8ef8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.401848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4460-dd4f-450e-a7ba-de6742c21848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.401943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3f9a1-27e5-496f-897b-ab07bb8a8ef8 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.402142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3f9a1-27e5-496f-897b-ab07bb8a8ef8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.402438 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d3f9a1-27e5-496f-897b-ab07bb8a8ef8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.402586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4460-dd4f-450e-a7ba-de6742c21848 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.402964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4460-dd4f-450e-a7ba-de6742c21848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.403314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdaf4460-dd4f-450e-a7ba-de6742c21848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.404030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf4460-dd4f-450e-a7ba-de6742c21848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.404672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-33d3f9a1-27e5-496f-897b-ab07bb8a8ef8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:08.406289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdaf4460-dd4f-450e-a7ba-de6742c21848 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.416897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:08.417734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1048/2069] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:08.423077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4460-dd4f-450e-a7ba-de6742c21848 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:08.436784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdaf4460-dd4f-450e-a7ba-de6742c21848 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.442612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c63f41e-4890-4721-beba-0abb9f5faea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.443370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f41e-4890-4721-beba-0abb9f5faea6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.443722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f41e-4890-4721-beba-0abb9f5faea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.444157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c63f41e-4890-4721-beba-0abb9f5faea6 admin admin] PATH_INFO: `/v3/projects/ac5e4f5e9f9e49f1b3ab45092a6e3013/users/292b0d583c4043028038e501df1793cf/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.461162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63f41e-4890-4721-beba-0abb9f5faea6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac5e4f5e9f9e49f1b3ab45092a6e3013, user_id=292b0d583c4043028038e501df1793cf, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.463985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c63f41e-4890-4721-beba-0abb9f5faea6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.478764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3f9a1-27e5-496f-897b-ab07bb8a8ef8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.479015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1049/2070] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:08 2022] PUT /identity/v3/projects/ac5e4f5e9f9e49f1b3ab45092a6e3013/users/292b0d583c4043028038e501df1793cf/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:08.480229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d3f9a1-27e5-496f-897b-ab07bb8a8ef8 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.489503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1022/2071] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:02:08.490383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6200b04-618f-44fe-81e7-eeb4cc68b01f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.490667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6200b04-618f-44fe-81e7-eeb4cc68b01f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.491021 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6200b04-618f-44fe-81e7-eeb4cc68b01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.518478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d6200b04-618f-44fe-81e7-eeb4cc68b01f None None] MFA Rules not processed for user `292b0d583c4043028038e501df1793cf`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:08.519745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d6200b04-618f-44fe-81e7-eeb4cc68b01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:08.591163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1050/2072] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:08 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:08.604362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3f9a1-27e5-496f-897b-ab07bb8a8ef8 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:08.613343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d3f9a1-27e5-496f-897b-ab07bb8a8ef8 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:08.664405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66682a9-59e8-44c8-9ea6-1065d2a5dd02 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:08.665029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66682a9-59e8-44c8-9ea6-1065d2a5dd02 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:08.665304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66682a9-59e8-44c8-9ea6-1065d2a5dd02 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:08.665567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66682a9-59e8-44c8-9ea6-1065d2a5dd02 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:08.673958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66682a9-59e8-44c8-9ea6-1065d2a5dd02 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:08.675444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66682a9-59e8-44c8-9ea6-1065d2a5dd02 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:08.684707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1023/2073] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:08 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:10.032836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6200b04-618f-44fe-81e7-eeb4cc68b01f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:10.044130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6200b04-618f-44fe-81e7-eeb4cc68b01f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:10.047675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:10.048338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:10.048547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:10.048815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:10.050659 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:10.114891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:10.116738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:10.127374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1051/2074] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:12.244426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66682a9-59e8-44c8-9ea6-1065d2a5dd02 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:12.253483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66682a9-59e8-44c8-9ea6-1065d2a5dd02 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:12.257669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f0c73a-cfed-4361-87a1-67a850ac8e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:12.258310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0c73a-cfed-4361-87a1-67a850ac8e68 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:12.258587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0c73a-cfed-4361-87a1-67a850ac8e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:12.258851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f0c73a-cfed-4361-87a1-67a850ac8e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:12.321164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0c73a-cfed-4361-87a1-67a850ac8e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:12.322665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f0c73a-cfed-4361-87a1-67a850ac8e68 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:12.332216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1024/2075] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:12 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:18.626348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:18.627868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:18.638240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:18.695924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5367058a-165d-48af-a64b-2f7c916c6fb6 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:18.700897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:18.701522 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:18.701791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:18.702062 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] PATH_INFO: `/v3/users/9b55115ba4594da9bb14a1731312113a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:18.703222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b55115ba4594da9bb14a1731312113a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:18.705137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:18.754926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b55115ba4594da9bb14a1731312113a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:18.755576 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b55115ba4594da9bb14a1731312113a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:18.765724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b55115ba4594da9bb14a1731312113a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:18.778686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b55115ba4594da9bb14a1731312113a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:18.779325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b55115ba4594da9bb14a1731312113a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:18.793433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1052/2076] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:18 2022] DELETE /identity/v3/users/9b55115ba4594da9bb14a1731312113a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:18.798054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a57e8-0206-42bf-9e47-f71644fbb9f0 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:18.798629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a57e8-0206-42bf-9e47-f71644fbb9f0 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:18.798727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998a57e8-0206-42bf-9e47-f71644fbb9f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:18.818500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-998a57e8-0206-42bf-9e47-f71644fbb9f0 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:18.819736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-998a57e8-0206-42bf-9e47-f71644fbb9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:18.863448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:18.880644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1025/2077] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:18.890600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:18.892310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:18.952377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c49f22-389a-4154-9e52-5aa23dbd7d77 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:18.956198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d5b8-5ab7-4dff-a906-f379360dfead service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:18.956816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d5b8-5ab7-4dff-a906-f379360dfead service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:18.957094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d5b8-5ab7-4dff-a906-f379360dfead service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:18.957366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e878d5b8-5ab7-4dff-a906-f379360dfead service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:18.966887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878d5b8-5ab7-4dff-a906-f379360dfead service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:18.968328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e878d5b8-5ab7-4dff-a906-f379360dfead service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:18.978121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1053/2078] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:19.259200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a57e8-0206-42bf-9e47-f71644fbb9f0 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:19.260899 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-998a57e8-0206-42bf-9e47-f71644fbb9f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:19.273737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998a57e8-0206-42bf-9e47-f71644fbb9f0 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:19.279021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:19.279670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:19.279946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:19.280215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] PATH_INFO: `/v3/projects/9863686340f4420c85b483f7ae31a747` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:19.281525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9863686340f4420c85b483f7ae31a747)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:19.283472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:19.334715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9863686340f4420c85b483f7ae31a747', 'request_id': 'req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.340968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9863686340f4420c85b483f7ae31a747', 'request_id': 'req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.347604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9863686340f4420c85b483f7ae31a747', 'request_id': 'req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.363781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] The token cache is being invalidate because project 9863686340f4420c85b483f7ae31a747 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:19.364072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.364773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.366448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1026/2079] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:19 2022] DELETE /identity/v3/projects/9863686340f4420c85b483f7ae31a747 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:19.370747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d5b8-5ab7-4dff-a906-f379360dfead service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:19.380423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e878d5b8-5ab7-4dff-a906-f379360dfead service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:19.407887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:19.438625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:19.439260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:19.439581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:19.439856 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] PATH_INFO: `/v3/users/ac8df33c65b94319b455c89d7a55c2e8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:19.440983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac8df33c65b94319b455c89d7a55c2e8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:19.442966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:19.490719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac8df33c65b94319b455c89d7a55c2e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.491441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac8df33c65b94319b455c89d7a55c2e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.500534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac8df33c65b94319b455c89d7a55c2e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.512956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac8df33c65b94319b455c89d7a55c2e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.513601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-bab346cb-7c02-4530-8fdb-3142949a4613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac8df33c65b94319b455c89d7a55c2e8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:19.526908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1054/2080] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:19 2022] DELETE /identity/v3/users/ac8df33c65b94319b455c89d7a55c2e8 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:19.866584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:19.868107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:19.882300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60c0c4c-5e64-44c6-a5b4-aa27504ced6a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:19.947766 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:19.948325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:19.948604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:19.948876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] PATH_INFO: `/v3/projects/747bd511d8124fa7bc986479168af4b0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:19.950030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=747bd511d8124fa7bc986479168af4b0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:19.951979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:20.009472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '747bd511d8124fa7bc986479168af4b0', 'request_id': 'req-53da8be0-99d9-408d-b433-ee86a8133eb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:20.016209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '747bd511d8124fa7bc986479168af4b0', 'request_id': 'req-53da8be0-99d9-408d-b433-ee86a8133eb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:20.023369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '747bd511d8124fa7bc986479168af4b0', 'request_id': 'req-53da8be0-99d9-408d-b433-ee86a8133eb2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:20.039684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] The token cache is being invalidate because project 747bd511d8124fa7bc986479168af4b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:20.040307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:20.041342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-53da8be0-99d9-408d-b433-ee86a8133eb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:20.043175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1027/2081] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:19 2022] DELETE /identity/v3/projects/747bd511d8124fa7bc986479168af4b0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:20.048639 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7495-a527-49a7-a5e0-845de6588e50 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.049148 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7495-a527-49a7-a5e0-845de6588e50 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.049229 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4b7495-a527-49a7-a5e0-845de6588e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.070245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fd4b7495-a527-49a7-a5e0-845de6588e50 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:20.071483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fd4b7495-a527-49a7-a5e0-845de6588e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:20.142035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1055/2082] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:20.150322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d927347-87a4-4bb8-85bd-3af2e6628817 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.150954 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d927347-87a4-4bb8-85bd-3af2e6628817 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.151231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d927347-87a4-4bb8-85bd-3af2e6628817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.159201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:20.173010 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7d927347-87a4-4bb8-85bd-3af2e6628817 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:20.174799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7d927347-87a4-4bb8-85bd-3af2e6628817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:20.183078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1028/2083] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:20.187871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b7495-a527-49a7-a5e0-845de6588e50 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:20.198347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4b7495-a527-49a7-a5e0-845de6588e50 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:20.202785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cab28b-20cb-4a3c-b933-3e1fb59dc838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:20.203419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cab28b-20cb-4a3c-b933-3e1fb59dc838 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.203642 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cab28b-20cb-4a3c-b933-3e1fb59dc838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.203891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cab28b-20cb-4a3c-b933-3e1fb59dc838 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.204539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cab28b-20cb-4a3c-b933-3e1fb59dc838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:20.204949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cab28b-20cb-4a3c-b933-3e1fb59dc838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:20.206584 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cab28b-20cb-4a3c-b933-3e1fb59dc838 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:20.218527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1056/2084] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:20.223052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d927347-87a4-4bb8-85bd-3af2e6628817 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:20.232663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d927347-87a4-4bb8-85bd-3af2e6628817 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:20.237489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa215b92-6194-4586-8228-1a16a7bffcb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:20.238116 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa215b92-6194-4586-8228-1a16a7bffcb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.238392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa215b92-6194-4586-8228-1a16a7bffcb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.238667 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa215b92-6194-4586-8228-1a16a7bffcb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.239327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa215b92-6194-4586-8228-1a16a7bffcb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:20.241315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa215b92-6194-4586-8228-1a16a7bffcb2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:20.272648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1029/2085] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:20 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:20.276615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cab28b-20cb-4a3c-b933-3e1fb59dc838 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:20.287421 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cab28b-20cb-4a3c-b933-3e1fb59dc838 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:20.328877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:20.357463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62189f8f-2ec0-4c11-bdc6-86733df6a316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:20.358103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62189f8f-2ec0-4c11-bdc6-86733df6a316 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.358419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62189f8f-2ec0-4c11-bdc6-86733df6a316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.358696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62189f8f-2ec0-4c11-bdc6-86733df6a316 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.359362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62189f8f-2ec0-4c11-bdc6-86733df6a316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:20.361335 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62189f8f-2ec0-4c11-bdc6-86733df6a316 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:20.385813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1057/2086] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:20 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:20.390486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa215b92-6194-4586-8228-1a16a7bffcb2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:20.403756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa215b92-6194-4586-8228-1a16a7bffcb2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:20.408613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33537623-5994-4894-a500-3e55cd917a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:20.409227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33537623-5994-4894-a500-3e55cd917a0e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.409503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33537623-5994-4894-a500-3e55cd917a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.409800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33537623-5994-4894-a500-3e55cd917a0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.410527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33537623-5994-4894-a500-3e55cd917a0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:20.412179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33537623-5994-4894-a500-3e55cd917a0e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:20.422768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1030/2087] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:20.426878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62189f8f-2ec0-4c11-bdc6-86733df6a316 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:20.439887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62189f8f-2ec0-4c11-bdc6-86733df6a316 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:20.448633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7743165d-763d-4e71-bb87-c64211d24522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:20.449509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743165d-763d-4e71-bb87-c64211d24522 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.449726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743165d-763d-4e71-bb87-c64211d24522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.450123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7743165d-763d-4e71-bb87-c64211d24522 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.450882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743165d-763d-4e71-bb87-c64211d24522 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:20.453300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7743165d-763d-4e71-bb87-c64211d24522 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:20.466751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1058/2088] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:20 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:20.471229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33537623-5994-4894-a500-3e55cd917a0e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:20.481959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33537623-5994-4894-a500-3e55cd917a0e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:20.487083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e85f3-c453-4c71-b16a-edd39c5d6623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:20.487734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e85f3-c453-4c71-b16a-edd39c5d6623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.488007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e85f3-c453-4c71-b16a-edd39c5d6623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.488283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e85f3-c453-4c71-b16a-edd39c5d6623 admin admin] PATH_INFO: `/v3/projects/fca4d40662594a3aaa112f030c9bc3b6/users/9d8c7ac10d28438dad17c679a11c00aa/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.504460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e85f3-c453-4c71-b16a-edd39c5d6623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fca4d40662594a3aaa112f030c9bc3b6, user_id=9d8c7ac10d28438dad17c679a11c00aa, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:20.507164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e85f3-c453-4c71-b16a-edd39c5d6623 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:20.523711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1031/2089] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:20 2022] PUT /identity/v3/projects/fca4d40662594a3aaa112f030c9bc3b6/users/9d8c7ac10d28438dad17c679a11c00aa/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:20.534336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67c118-2c14-4971-9d7b-7cb93f72f96d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.534631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67c118-2c14-4971-9d7b-7cb93f72f96d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.534928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d67c118-2c14-4971-9d7b-7cb93f72f96d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.564589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1d67c118-2c14-4971-9d7b-7cb93f72f96d None None] MFA Rules not processed for user `9d8c7ac10d28438dad17c679a11c00aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:20.565814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1d67c118-2c14-4971-9d7b-7cb93f72f96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:20.645873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1059/2090] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:20.660263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e85f3-c453-4c71-b16a-edd39c5d6623 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:20.663317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-233e85f3-c453-4c71-b16a-edd39c5d6623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:20.730505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e85f3-c453-4c71-b16a-edd39c5d6623 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:20.735451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1e9509-67ec-47f6-9fb0-0b9f2cbd0ceb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:20.736073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9509-67ec-47f6-9fb0-0b9f2cbd0ceb service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:20.736358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9509-67ec-47f6-9fb0-0b9f2cbd0ceb service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:20.736622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1e9509-67ec-47f6-9fb0-0b9f2cbd0ceb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:20.746313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e9509-67ec-47f6-9fb0-0b9f2cbd0ceb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:20.747761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1e9509-67ec-47f6-9fb0-0b9f2cbd0ceb service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:20.756688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1032/2091] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:23.817812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67c118-2c14-4971-9d7b-7cb93f72f96d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:23.819334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1d67c118-2c14-4971-9d7b-7cb93f72f96d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:23.885502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d67c118-2c14-4971-9d7b-7cb93f72f96d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:23.891168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:23.891820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:23.892102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:23.892377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] PATH_INFO: `/v3/users/7803cc1f848442f9be04ba265d444530` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:23.893570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7803cc1f848442f9be04ba265d444530)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:23.895557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:23.946549 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:23.948295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7803cc1f848442f9be04ba265d444530'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:23.949052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7803cc1f848442f9be04ba265d444530'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:23.958971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7803cc1f848442f9be04ba265d444530'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:23.972137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7803cc1f848442f9be04ba265d444530'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:23.972778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7803cc1f848442f9be04ba265d444530'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:23.985038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1060/2092] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:23 2022] DELETE /identity/v3/users/7803cc1f848442f9be04ba265d444530 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:23.989638 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798026c-30e0-4f74-b4b7-ca709eef9e6e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:23.990237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798026c-30e0-4f74-b4b7-ca709eef9e6e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:23.990237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7798026c-30e0-4f74-b4b7-ca709eef9e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:24.009971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7798026c-30e0-4f74-b4b7-ca709eef9e6e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:24.011198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7798026c-30e0-4f74-b4b7-ca709eef9e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:24.074445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1033/2093] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:24.084162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:24.085810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:24.150040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184045c5-b1ae-4941-9645-c1d0b8a8c36a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:24.154613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7972be-431c-486f-b669-3475065ce88d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:24.155235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7972be-431c-486f-b669-3475065ce88d service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:24.155743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7972be-431c-486f-b669-3475065ce88d service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:24.155847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7972be-431c-486f-b669-3475065ce88d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:24.165730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7972be-431c-486f-b669-3475065ce88d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:24.167503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7972be-431c-486f-b669-3475065ce88d service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:24.177759 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1061/2094] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:24.527629 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798026c-30e0-4f74-b4b7-ca709eef9e6e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:24.529356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7798026c-30e0-4f74-b4b7-ca709eef9e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:24.542549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7798026c-30e0-4f74-b4b7-ca709eef9e6e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:24.547907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:24.548639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:24.548934 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:24.549208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] PATH_INFO: `/v3/projects/f69e7bcc7db34476b689772070c2f726` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:24.550584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f69e7bcc7db34476b689772070c2f726)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:24.552850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:24.603133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:24.614636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f69e7bcc7db34476b689772070c2f726', 'request_id': 'req-f9006ff4-b3f6-4856-a736-402732132e97'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.625539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f69e7bcc7db34476b689772070c2f726', 'request_id': 'req-f9006ff4-b3f6-4856-a736-402732132e97'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.632470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f69e7bcc7db34476b689772070c2f726', 'request_id': 'req-f9006ff4-b3f6-4856-a736-402732132e97'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.648189 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] The token cache is being invalidate because project f69e7bcc7db34476b689772070c2f726 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:24.648486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.649113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.650719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1034/2095] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:24 2022] DELETE /identity/v3/projects/f69e7bcc7db34476b689772070c2f726 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:24.654918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7972be-431c-486f-b669-3475065ce88d service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:24.664725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7972be-431c-486f-b669-3475065ce88d service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:24.724028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:24.724670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:24.724944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:24.725211 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] PATH_INFO: `/v3/users/33abb44c09424991a1fc169260377662` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:24.726357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33abb44c09424991a1fc169260377662)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:24.728303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:24.782935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33abb44c09424991a1fc169260377662'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.783605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33abb44c09424991a1fc169260377662'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.791868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33abb44c09424991a1fc169260377662'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.803399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:24.806495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33abb44c09424991a1fc169260377662'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.807344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33abb44c09424991a1fc169260377662'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:24.821241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1062/2096] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:24 2022] DELETE /identity/v3/users/33abb44c09424991a1fc169260377662 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:25.107222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:25.108792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:25.123956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9006ff4-b3f6-4856-a736-402732132e97 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:25.181157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:25.181801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:25.182100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:25.182446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] PATH_INFO: `/v3/projects/0bd8d04167d64e99891a4d01faf7f518` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:25.183747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0bd8d04167d64e99891a4d01faf7f518)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:25.185642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:25.236469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0bd8d04167d64e99891a4d01faf7f518', 'request_id': 'req-1b53672d-91ad-4099-b68b-ff059bff7ee4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.242679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0bd8d04167d64e99891a4d01faf7f518', 'request_id': 'req-1b53672d-91ad-4099-b68b-ff059bff7ee4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.249038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0bd8d04167d64e99891a4d01faf7f518', 'request_id': 'req-1b53672d-91ad-4099-b68b-ff059bff7ee4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.264482 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] The token cache is being invalidate because project 0bd8d04167d64e99891a4d01faf7f518 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:25.264765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.265455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.267045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1035/2097] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:25 2022] DELETE /identity/v3/projects/0bd8d04167d64e99891a4d01faf7f518 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:25.271665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:25.273795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:25.288155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e60d9a-c6a7-4f63-beea-d364b12851a8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:25.355610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:25.356244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:25.356501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:25.356797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] PATH_INFO: `/v3/users/a11a5ff1a3d54379baa7a4aa6fe84cb2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:25.358072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11a5ff1a3d54379baa7a4aa6fe84cb2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:25.360054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:25.412774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11a5ff1a3d54379baa7a4aa6fe84cb2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.413521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11a5ff1a3d54379baa7a4aa6fe84cb2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.424155 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11a5ff1a3d54379baa7a4aa6fe84cb2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.439188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11a5ff1a3d54379baa7a4aa6fe84cb2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.439764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a26f239-626b-45ab-8b38-be1a8afcc8d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11a5ff1a3d54379baa7a4aa6fe84cb2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.452479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1063/2098] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:25 2022] DELETE /identity/v3/users/a11a5ff1a3d54379baa7a4aa6fe84cb2 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:25.730396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:25.731967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:25.748041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b53672d-91ad-4099-b68b-ff059bff7ee4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:25.802332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:25.811379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:25.812041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:25.812619 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:25.812980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] PATH_INFO: `/v3/projects/a0c95c90475e4f6caa4d5cbf7b33e356` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:25.814453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0c95c90475e4f6caa4d5cbf7b33e356)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:25.816428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:25.865267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0c95c90475e4f6caa4d5cbf7b33e356', 'request_id': 'req-62cff54f-11fd-4831-90b5-7406133ca8cc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.871675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0c95c90475e4f6caa4d5cbf7b33e356', 'request_id': 'req-62cff54f-11fd-4831-90b5-7406133ca8cc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.878141 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0c95c90475e4f6caa4d5cbf7b33e356', 'request_id': 'req-62cff54f-11fd-4831-90b5-7406133ca8cc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.892778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] The token cache is being invalidate because project a0c95c90475e4f6caa4d5cbf7b33e356 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:25.893070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.893742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-62cff54f-11fd-4831-90b5-7406133ca8cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:25.895300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1036/2099] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:25 2022] DELETE /identity/v3/projects/a0c95c90475e4f6caa4d5cbf7b33e356 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:25.900613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c9b00-3a04-49c2-9b51-ea845ad37606 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:25.901109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c9b00-3a04-49c2-9b51-ea845ad37606 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:25.901258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53c9b00-3a04-49c2-9b51-ea845ad37606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:25.921544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d53c9b00-3a04-49c2-9b51-ea845ad37606 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:25.922774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d53c9b00-3a04-49c2-9b51-ea845ad37606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:25.986695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1064/2100] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:25.997336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075561f-2327-4b60-9b80-ae9d1c0ba155 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:25.997336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075561f-2327-4b60-9b80-ae9d1c0ba155 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:25.997336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0075561f-2327-4b60-9b80-ae9d1c0ba155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.017908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0075561f-2327-4b60-9b80-ae9d1c0ba155 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:26.019138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0075561f-2327-4b60-9b80-ae9d1c0ba155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:26.027285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1037/2101] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:26.031698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c9b00-3a04-49c2-9b51-ea845ad37606 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.038053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:26.044368 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53c9b00-3a04-49c2-9b51-ea845ad37606 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.051732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a50a7-981a-425b-a584-8651fb8edd4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.052580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a50a7-981a-425b-a584-8651fb8edd4c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.053169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a50a7-981a-425b-a584-8651fb8edd4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.053439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693a50a7-981a-425b-a584-8651fb8edd4c admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.054101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a50a7-981a-425b-a584-8651fb8edd4c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:26.054451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a50a7-981a-425b-a584-8651fb8edd4c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.056431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693a50a7-981a-425b-a584-8651fb8edd4c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.072303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1065/2102] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:26.073640 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075561f-2327-4b60-9b80-ae9d1c0ba155 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.082855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0075561f-2327-4b60-9b80-ae9d1c0ba155 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.087583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fba129-c210-4075-8e71-34e27350e264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.088184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fba129-c210-4075-8e71-34e27350e264 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.088461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fba129-c210-4075-8e71-34e27350e264 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.089215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fba129-c210-4075-8e71-34e27350e264 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.089917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fba129-c210-4075-8e71-34e27350e264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.091873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fba129-c210-4075-8e71-34e27350e264 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.122671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1038/2103] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:26 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:26.126997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a50a7-981a-425b-a584-8651fb8edd4c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.137208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693a50a7-981a-425b-a584-8651fb8edd4c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.196813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7c917-accf-43db-b88c-f0035e97cbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.197692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7c917-accf-43db-b88c-f0035e97cbee admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.198267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7c917-accf-43db-b88c-f0035e97cbee admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.198635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7c917-accf-43db-b88c-f0035e97cbee admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.199223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7c917-accf-43db-b88c-f0035e97cbee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.201142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7c917-accf-43db-b88c-f0035e97cbee admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.224246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1066/2104] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:26 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:26.228378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fba129-c210-4075-8e71-34e27350e264 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.239037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fba129-c210-4075-8e71-34e27350e264 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.244361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bece1b-7cf8-474c-92eb-09495c1994e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.244979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bece1b-7cf8-474c-92eb-09495c1994e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.245336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bece1b-7cf8-474c-92eb-09495c1994e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.245658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bece1b-7cf8-474c-92eb-09495c1994e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.246366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bece1b-7cf8-474c-92eb-09495c1994e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.248018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12bece1b-7cf8-474c-92eb-09495c1994e7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.258672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1039/2105] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:26.262574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7c917-accf-43db-b88c-f0035e97cbee admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.272720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7c917-accf-43db-b88c-f0035e97cbee admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.278624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54f63be-4b22-4700-b520-e830bd3c5843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.279246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f63be-4b22-4700-b520-e830bd3c5843 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.279507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f63be-4b22-4700-b520-e830bd3c5843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.279778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54f63be-4b22-4700-b520-e830bd3c5843 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.280551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f63be-4b22-4700-b520-e830bd3c5843 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.282285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54f63be-4b22-4700-b520-e830bd3c5843 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.293978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1067/2106] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:26.298993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bece1b-7cf8-474c-92eb-09495c1994e7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.312395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bece1b-7cf8-474c-92eb-09495c1994e7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.317634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9428e-65f9-4578-a2ab-e1c8af8011a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.318345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9428e-65f9-4578-a2ab-e1c8af8011a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.318683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9428e-65f9-4578-a2ab-e1c8af8011a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.318962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e9428e-65f9-4578-a2ab-e1c8af8011a3 admin admin] PATH_INFO: `/v3/projects/496f4307ecfb4214bd705e3a9a8408ed/users/dd8e8bd1582145608a8da684a103c94e/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.342866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9428e-65f9-4578-a2ab-e1c8af8011a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=496f4307ecfb4214bd705e3a9a8408ed, user_id=dd8e8bd1582145608a8da684a103c94e, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.345663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e9428e-65f9-4578-a2ab-e1c8af8011a3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.361071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1040/2107] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:26 2022] PUT /identity/v3/projects/496f4307ecfb4214bd705e3a9a8408ed/users/dd8e8bd1582145608a8da684a103c94e/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:26.372746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca72fc-3d7e-4f43-b270-83449e76a82b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.373093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca72fc-3d7e-4f43-b270-83449e76a82b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.373365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cca72fc-3d7e-4f43-b270-83449e76a82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.406122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7cca72fc-3d7e-4f43-b270-83449e76a82b None None] MFA Rules not processed for user `dd8e8bd1582145608a8da684a103c94e`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:26.407477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7cca72fc-3d7e-4f43-b270-83449e76a82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:26.496568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1068/2108] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:26.501190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9428e-65f9-4578-a2ab-e1c8af8011a3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.571897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e9428e-65f9-4578-a2ab-e1c8af8011a3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.579758 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490addd6-ad66-4df5-bab0-7359bec986f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.580419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490addd6-ad66-4df5-bab0-7359bec986f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.580705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490addd6-ad66-4df5-bab0-7359bec986f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.580963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490addd6-ad66-4df5-bab0-7359bec986f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.582353 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490addd6-ad66-4df5-bab0-7359bec986f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.585115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490addd6-ad66-4df5-bab0-7359bec986f5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.629364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1041/2109] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:26 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:26.630808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca72fc-3d7e-4f43-b270-83449e76a82b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.642538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cca72fc-3d7e-4f43-b270-83449e76a82b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.699830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b618f-c32c-480a-bdd9-329344709001 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.700446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b618f-c32c-480a-bdd9-329344709001 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.700722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b618f-c32c-480a-bdd9-329344709001 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.700991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b618f-c32c-480a-bdd9-329344709001 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.701665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b618f-c32c-480a-bdd9-329344709001 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.703602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b618f-c32c-480a-bdd9-329344709001 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.711562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:26.728781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1069/2110] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:26 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:26.733895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490addd6-ad66-4df5-bab0-7359bec986f5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.748479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490addd6-ad66-4df5-bab0-7359bec986f5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.753489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fbcc7-c384-4405-b795-a6452b319b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.754145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fbcc7-c384-4405-b795-a6452b319b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.754422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fbcc7-c384-4405-b795-a6452b319b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.754692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592fbcc7-c384-4405-b795-a6452b319b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.755378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fbcc7-c384-4405-b795-a6452b319b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.757137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592fbcc7-c384-4405-b795-a6452b319b18 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.759334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:26.774844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1042/2111] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:26.778579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b618f-c32c-480a-bdd9-329344709001 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.788933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b618f-c32c-480a-bdd9-329344709001 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.794028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1e586e-44d6-42f6-801a-d937bee4cf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.794681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e586e-44d6-42f6-801a-d937bee4cf71 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.794960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e586e-44d6-42f6-801a-d937bee4cf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.795232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1e586e-44d6-42f6-801a-d937bee4cf71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.795877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e586e-44d6-42f6-801a-d937bee4cf71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.797493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1e586e-44d6-42f6-801a-d937bee4cf71 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.808267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1070/2112] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:26.812891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fbcc7-c384-4405-b795-a6452b319b18 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:26.822990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592fbcc7-c384-4405-b795-a6452b319b18 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:26.828800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48929198-8a08-48b4-9307-a67479043b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:26.829466 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48929198-8a08-48b4-9307-a67479043b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.829745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48929198-8a08-48b4-9307-a67479043b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.830018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48929198-8a08-48b4-9307-a67479043b3f admin admin] PATH_INFO: `/v3/projects/487ce54f65164d799b7f1c21eb8580a1/users/3e776fc3ce3449788986e1282ec1bb6f/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.845525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48929198-8a08-48b4-9307-a67479043b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=487ce54f65164d799b7f1c21eb8580a1, user_id=3e776fc3ce3449788986e1282ec1bb6f, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:26.848226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48929198-8a08-48b4-9307-a67479043b3f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:26.863519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1043/2113] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:26 2022] PUT /identity/v3/projects/487ce54f65164d799b7f1c21eb8580a1/users/3e776fc3ce3449788986e1282ec1bb6f/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:26.875336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b913e0-6509-4a56-8ad0-203a65862b46 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:26.875715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b913e0-6509-4a56-8ad0-203a65862b46 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:26.876074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b913e0-6509-4a56-8ad0-203a65862b46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:26.908214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-45b913e0-6509-4a56-8ad0-203a65862b46 None None] MFA Rules not processed for user `3e776fc3ce3449788986e1282ec1bb6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:26.909553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-45b913e0-6509-4a56-8ad0-203a65862b46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:26.984528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1071/2114] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:26 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:26.989108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48929198-8a08-48b4-9307-a67479043b3f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:27.062808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48929198-8a08-48b4-9307-a67479043b3f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:27.067632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32bdcf-3222-418e-b331-40f03b322796 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:27.068245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32bdcf-3222-418e-b331-40f03b322796 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:27.068527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32bdcf-3222-418e-b331-40f03b322796 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:27.068796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32bdcf-3222-418e-b331-40f03b322796 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:27.069428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32bdcf-3222-418e-b331-40f03b322796 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:27.071379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f32bdcf-3222-418e-b331-40f03b322796 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:27.100935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1044/2115] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:26 2022] POST /identity/v3/projects => generated 383 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:27.105251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b913e0-6509-4a56-8ad0-203a65862b46 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:27.116683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b913e0-6509-4a56-8ad0-203a65862b46 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:27.175334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acf8d9-e655-4a97-bead-2323c6881911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:27.175864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acf8d9-e655-4a97-bead-2323c6881911 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:27.176145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acf8d9-e655-4a97-bead-2323c6881911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:27.176417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acf8d9-e655-4a97-bead-2323c6881911 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:27.177059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acf8d9-e655-4a97-bead-2323c6881911 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:27.179028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acf8d9-e655-4a97-bead-2323c6881911 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:27.203315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1072/2116] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:27 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:27.207620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32bdcf-3222-418e-b331-40f03b322796 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:27.218759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32bdcf-3222-418e-b331-40f03b322796 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:27.223485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db241b-6df5-450b-91b8-06efa9309cb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:27.224092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db241b-6df5-450b-91b8-06efa9309cb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:27.224374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db241b-6df5-450b-91b8-06efa9309cb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:27.224645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86db241b-6df5-450b-91b8-06efa9309cb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:27.225293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db241b-6df5-450b-91b8-06efa9309cb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:27.226933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86db241b-6df5-450b-91b8-06efa9309cb1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:27.237956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1045/2117] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:27.242233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acf8d9-e655-4a97-bead-2323c6881911 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:27.253161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acf8d9-e655-4a97-bead-2323c6881911 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:27.257925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c730d17-6d56-4e78-b69a-d746ed987a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:27.258626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c730d17-6d56-4e78-b69a-d746ed987a32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:27.258904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c730d17-6d56-4e78-b69a-d746ed987a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:27.259204 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c730d17-6d56-4e78-b69a-d746ed987a32 admin admin] PATH_INFO: `/v3/projects/982d94fa54754ca7b12cfa4b3916eb44/users/cc7f09f62ba44400a52e52985f2ed3e0/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:27.275221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c730d17-6d56-4e78-b69a-d746ed987a32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=982d94fa54754ca7b12cfa4b3916eb44, user_id=cc7f09f62ba44400a52e52985f2ed3e0, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:27.278128 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c730d17-6d56-4e78-b69a-d746ed987a32 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:27.294631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1073/2118] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:27 2022] PUT /identity/v3/projects/982d94fa54754ca7b12cfa4b3916eb44/users/cc7f09f62ba44400a52e52985f2ed3e0/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:27.298870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db241b-6df5-450b-91b8-06efa9309cb1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:27.311076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86db241b-6df5-450b-91b8-06efa9309cb1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:27.370437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b123660c-eb82-41e7-8ff2-eea278595b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:27.371073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123660c-eb82-41e7-8ff2-eea278595b30 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:27.371350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123660c-eb82-41e7-8ff2-eea278595b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:27.371621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b123660c-eb82-41e7-8ff2-eea278595b30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:27.372297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123660c-eb82-41e7-8ff2-eea278595b30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:27.373937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b123660c-eb82-41e7-8ff2-eea278595b30 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:27.384488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1046/2119] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:27 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:27.388973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c730d17-6d56-4e78-b69a-d746ed987a32 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:27.399443 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c730d17-6d56-4e78-b69a-d746ed987a32 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:27.404312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151d87-aced-4f10-b854-3a8ec2a61666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:27.404967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151d87-aced-4f10-b854-3a8ec2a61666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:27.405242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151d87-aced-4f10-b854-3a8ec2a61666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:27.405510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0151d87-aced-4f10-b854-3a8ec2a61666 admin admin] PATH_INFO: `/v3/projects/982d94fa54754ca7b12cfa4b3916eb44/users/cc7f09f62ba44400a52e52985f2ed3e0/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:27.407446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0151d87-aced-4f10-b854-3a8ec2a61666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=982d94fa54754ca7b12cfa4b3916eb44, user_id=cc7f09f62ba44400a52e52985f2ed3e0, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:27.410230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0151d87-aced-4f10-b854-3a8ec2a61666 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:27.421469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1074/2120] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:27 2022] PUT /identity/v3/projects/982d94fa54754ca7b12cfa4b3916eb44/users/cc7f09f62ba44400a52e52985f2ed3e0/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:27.432502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32fded-0c1e-4d06-aa67-6e7cdc733bdf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:27.432898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32fded-0c1e-4d06-aa67-6e7cdc733bdf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:27.433304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db32fded-0c1e-4d06-aa67-6e7cdc733bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:27.464238 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-db32fded-0c1e-4d06-aa67-6e7cdc733bdf None None] MFA Rules not processed for user `cc7f09f62ba44400a52e52985f2ed3e0`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:27.465442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-db32fded-0c1e-4d06-aa67-6e7cdc733bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:27.503992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:27.547569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1047/2121] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:27 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:27.559571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151d87-aced-4f10-b854-3a8ec2a61666 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:27.561411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d0151d87-aced-4f10-b854-3a8ec2a61666 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:27.585032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:27.623477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0151d87-aced-4f10-b854-3a8ec2a61666 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:27.627326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ef222e-55bc-417a-9dcd-29cc0c958198 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:27.627944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef222e-55bc-417a-9dcd-29cc0c958198 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:27.628227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef222e-55bc-417a-9dcd-29cc0c958198 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:27.628499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ef222e-55bc-417a-9dcd-29cc0c958198 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:27.692215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef222e-55bc-417a-9dcd-29cc0c958198 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:27.693692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ef222e-55bc-417a-9dcd-29cc0c958198 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:27.704218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1075/2122] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:27.754045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539e119-8bf2-4221-95ff-b3063b5baff7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:27.754433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539e119-8bf2-4221-95ff-b3063b5baff7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:27.755441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539e119-8bf2-4221-95ff-b3063b5baff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:27.775593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5539e119-8bf2-4221-95ff-b3063b5baff7 None None] MFA Rules not processed for user `5bd897d855c64ae28ec3f10f69e9e8b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:27.776937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5539e119-8bf2-4221-95ff-b3063b5baff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:27.837095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1048/2123] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:27 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:32.020082 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbc625-dbb4-4834-b284-458cb1f2ec7a None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:32.020726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbc625-dbb4-4834-b284-458cb1f2ec7a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:32.020726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfbc625-dbb4-4834-b284-458cb1f2ec7a None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:32.022658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1076/2124] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:02:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:02:32.038660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bcc55-0460-4667-96fe-acda77b53a11 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:32.039174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bcc55-0460-4667-96fe-acda77b53a11 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:32.039427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69bcc55-0460-4667-96fe-acda77b53a11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:32.066265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b69bcc55-0460-4667-96fe-acda77b53a11 None None] MFA Rules not processed for user `e8202ad6007246078c4d25365dc5a0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:32.067566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b69bcc55-0460-4667-96fe-acda77b53a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:32.126703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbc625-dbb4-4834-b284-458cb1f2ec7a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:32.128689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5bfbc625-dbb4-4834-b284-458cb1f2ec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:32.184720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1049/2125] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:02:32 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:32.196885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bcc55-0460-4667-96fe-acda77b53a11 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:32.198488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b69bcc55-0460-4667-96fe-acda77b53a11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:32.216066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69bcc55-0460-4667-96fe-acda77b53a11 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:32.221837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3ac78-44a6-427b-a795-ff9f51ff44dc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:32.222622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3ac78-44a6-427b-a795-ff9f51ff44dc service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:32.223175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3ac78-44a6-427b-a795-ff9f51ff44dc service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:32.223855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb3ac78-44a6-427b-a795-ff9f51ff44dc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:32.236070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3ac78-44a6-427b-a795-ff9f51ff44dc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:32.239057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfbc625-dbb4-4834-b284-458cb1f2ec7a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:32.242568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb3ac78-44a6-427b-a795-ff9f51ff44dc service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:32.245555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:32.247610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:32.248060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:32.248495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] PATH_INFO: `/v3/users/23088396d0ca47c5929690820a3f7e29` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:32.250115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] RBAC: Authorizing `identity:delete_user(user_id=23088396d0ca47c5929690820a3f7e29)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:32.252130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:32.264714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1050/2126] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:32 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:02:32.330655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23088396d0ca47c5929690820a3f7e29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.331338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23088396d0ca47c5929690820a3f7e29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.341647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23088396d0ca47c5929690820a3f7e29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.361941 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23088396d0ca47c5929690820a3f7e29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.362826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23088396d0ca47c5929690820a3f7e29'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.368319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:32.387120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1077/2127] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:32 2022] DELETE /identity/v3/users/23088396d0ca47c5929690820a3f7e29 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:32.562505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3ac78-44a6-427b-a795-ff9f51ff44dc service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:32.579105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb3ac78-44a6-427b-a795-ff9f51ff44dc service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:32.639123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:32.639752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:32.640034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:32.640360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:32.642955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:32.653352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:32.675912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:32.677732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:32.720896 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:32.722416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:32.731557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1051/2128] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:32.733445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:32.734474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:32.786195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcda504-f3d2-4e87-ae63-522840e57efb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:32.791436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:32.792164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:32.792438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:32.792707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] PATH_INFO: `/v3/projects/5d415383e1c1471aae5712893d48c2b6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:32.794508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d415383e1c1471aae5712893d48c2b6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:32.798784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:32.813214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44970dd-9a78-4d47-bced-302d623df5d2 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:32.819422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:32.820609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:32.820904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:32.821188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] PATH_INFO: `/v3/users/9d8c7ac10d28438dad17c679a11c00aa` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:32.822733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9d8c7ac10d28438dad17c679a11c00aa)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:32.825645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:32.856359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d415383e1c1471aae5712893d48c2b6', 'request_id': 'req-809ab2b2-a66c-4481-8b49-408f2c4e6f47'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.863688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d415383e1c1471aae5712893d48c2b6', 'request_id': 'req-809ab2b2-a66c-4481-8b49-408f2c4e6f47'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.870448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d415383e1c1471aae5712893d48c2b6', 'request_id': 'req-809ab2b2-a66c-4481-8b49-408f2c4e6f47'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.887674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9d8c7ac10d28438dad17c679a11c00aa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.890143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] The token cache is being invalidate because project 5d415383e1c1471aae5712893d48c2b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:32.890447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.891196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-809ab2b2-a66c-4481-8b49-408f2c4e6f47 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.892914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1078/2129] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:32 2022] DELETE /identity/v3/projects/5d415383e1c1471aae5712893d48c2b6 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:32.897333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9d8c7ac10d28438dad17c679a11c00aa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.898313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9d851-4f47-4524-8690-48275373f17f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:32.898722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9d851-4f47-4524-8690-48275373f17f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:32.898906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f9d851-4f47-4524-8690-48275373f17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:32.912364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9d8c7ac10d28438dad17c679a11c00aa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.913079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9d8c7ac10d28438dad17c679a11c00aa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.921278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-723249cf-221d-4cb6-855d-f1d20b177045 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9d8c7ac10d28438dad17c679a11c00aa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:32.921429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f1f9d851-4f47-4524-8690-48275373f17f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:32.922578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f1f9d851-4f47-4524-8690-48275373f17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:32.931300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1052/2130] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:32 2022] DELETE /identity/v3/users/9d8c7ac10d28438dad17c679a11c00aa => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:32.936191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18eb8b-4443-4cd9-8cfa-e145703a0051 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:32.936654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18eb8b-4443-4cd9-8cfa-e145703a0051 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:32.937077 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf18eb8b-4443-4cd9-8cfa-e145703a0051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:32.958443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bf18eb8b-4443-4cd9-8cfa-e145703a0051 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:32.963269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf18eb8b-4443-4cd9-8cfa-e145703a0051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:32.995109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1079/2131] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:33.003122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff8a9e-3100-4f0d-ab3a-8494be06a7b3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.003552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff8a9e-3100-4f0d-ab3a-8494be06a7b3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.004851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ff8a9e-3100-4f0d-ab3a-8494be06a7b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.040581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:33.046734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f1ff8a9e-3100-4f0d-ab3a-8494be06a7b3 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:33.048650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f1ff8a9e-3100-4f0d-ab3a-8494be06a7b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:33.095051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:33.100381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1053/2132] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:33.125372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18eb8b-4443-4cd9-8cfa-e145703a0051 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:33.127308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf18eb8b-4443-4cd9-8cfa-e145703a0051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:33.174253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1080/2133] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:33.179153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff8a9e-3100-4f0d-ab3a-8494be06a7b3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:33.215118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ff8a9e-3100-4f0d-ab3a-8494be06a7b3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:33.230272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09fbfb-d28e-40fb-b26f-637d50c6abf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:33.231102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09fbfb-d28e-40fb-b26f-637d50c6abf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.231533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09fbfb-d28e-40fb-b26f-637d50c6abf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.232030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae09fbfb-d28e-40fb-b26f-637d50c6abf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.232660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09fbfb-d28e-40fb-b26f-637d50c6abf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:33.232990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09fbfb-d28e-40fb-b26f-637d50c6abf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:33.234907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae09fbfb-d28e-40fb-b26f-637d50c6abf4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:33.251369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1081/2134] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:33.255718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09fbfb-d28e-40fb-b26f-637d50c6abf4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:33.270452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae09fbfb-d28e-40fb-b26f-637d50c6abf4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:33.272763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf18eb8b-4443-4cd9-8cfa-e145703a0051 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:33.276500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399efef7-e788-4402-a010-3b9ff333151d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:33.277519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399efef7-e788-4402-a010-3b9ff333151d admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.277805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399efef7-e788-4402-a010-3b9ff333151d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.278101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399efef7-e788-4402-a010-3b9ff333151d admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.278883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399efef7-e788-4402-a010-3b9ff333151d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:33.280061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bd145-8f0c-4e06-bffc-200925d8748a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:33.280809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399efef7-e788-4402-a010-3b9ff333151d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:33.281224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bd145-8f0c-4e06-bffc-200925d8748a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.281719 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bd145-8f0c-4e06-bffc-200925d8748a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.282239 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081bd145-8f0c-4e06-bffc-200925d8748a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.296713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bd145-8f0c-4e06-bffc-200925d8748a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:33.299830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081bd145-8f0c-4e06-bffc-200925d8748a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:33.316321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1054/2135] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:33.323193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1082/2136] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:33 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:33.324117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bd145-8f0c-4e06-bffc-200925d8748a service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:33.337440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081bd145-8f0c-4e06-bffc-200925d8748a service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:33.411946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7b104-1583-4bc4-b0a8-7fa0439180c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:33.413026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7b104-1583-4bc4-b0a8-7fa0439180c9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.413486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7b104-1583-4bc4-b0a8-7fa0439180c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.413896 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee7b104-1583-4bc4-b0a8-7fa0439180c9 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.414694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7b104-1583-4bc4-b0a8-7fa0439180c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:33.417188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee7b104-1583-4bc4-b0a8-7fa0439180c9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:33.454913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1055/2137] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:33 2022] POST /identity/v3/users => generated 285 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:33.458922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399efef7-e788-4402-a010-3b9ff333151d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:33.474719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399efef7-e788-4402-a010-3b9ff333151d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:33.480189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b02e7c-ca77-4871-ba4b-1c8e742df09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:33.480883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02e7c-ca77-4871-ba4b-1c8e742df09d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.481165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02e7c-ca77-4871-ba4b-1c8e742df09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.481437 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b02e7c-ca77-4871-ba4b-1c8e742df09d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.482118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b02e7c-ca77-4871-ba4b-1c8e742df09d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:33.484332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b02e7c-ca77-4871-ba4b-1c8e742df09d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:33.500036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1083/2138] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:33.501299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7b104-1583-4bc4-b0a8-7fa0439180c9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:33.514548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee7b104-1583-4bc4-b0a8-7fa0439180c9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:33.521047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c1dfb-9c31-4b6c-acae-bf9c193a8c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:33.521694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1dfb-9c31-4b6c-acae-bf9c193a8c6a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.521979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1dfb-9c31-4b6c-acae-bf9c193a8c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.522291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83c1dfb-9c31-4b6c-acae-bf9c193a8c6a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.522941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c1dfb-9c31-4b6c-acae-bf9c193a8c6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:33.524786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83c1dfb-9c31-4b6c-acae-bf9c193a8c6a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:33.539191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b02e7c-ca77-4871-ba4b-1c8e742df09d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:33.541333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-65b02e7c-ca77-4871-ba4b-1c8e742df09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:33.546310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1056/2139] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:33 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:33.547425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c1dfb-9c31-4b6c-acae-bf9c193a8c6a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:33.558346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83c1dfb-9c31-4b6c-acae-bf9c193a8c6a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:33.563297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfe8020-cd28-47e7-b103-0605bcf77011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:33.563970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8020-cd28-47e7-b103-0605bcf77011 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.564284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8020-cd28-47e7-b103-0605bcf77011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.564569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfe8020-cd28-47e7-b103-0605bcf77011 admin admin] PATH_INFO: `/v3/projects/9d540eb551714dff92b908017ccfcad7/users/722c3c0f10694dcf99911d25bd6164a7/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.584857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8020-cd28-47e7-b103-0605bcf77011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9d540eb551714dff92b908017ccfcad7, user_id=722c3c0f10694dcf99911d25bd6164a7, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:33.589283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfe8020-cd28-47e7-b103-0605bcf77011 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:33.634802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b02e7c-ca77-4871-ba4b-1c8e742df09d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:33.656329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1057/2140] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:33 2022] PUT /identity/v3/projects/9d540eb551714dff92b908017ccfcad7/users/722c3c0f10694dcf99911d25bd6164a7/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:33.661303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebff31c-cb65-4abf-9cf7-1b4478b0eacb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.661677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebff31c-cb65-4abf-9cf7-1b4478b0eacb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.662380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebff31c-cb65-4abf-9cf7-1b4478b0eacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.727125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80061b0f-601f-41a2-aeb0-f543b635a205 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:33.727751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80061b0f-601f-41a2-aeb0-f543b635a205 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:33.728027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80061b0f-601f-41a2-aeb0-f543b635a205 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:33.729528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80061b0f-601f-41a2-aeb0-f543b635a205 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:33.731708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-80061b0f-601f-41a2-aeb0-f543b635a205 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:33.733051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8ebff31c-cb65-4abf-9cf7-1b4478b0eacb None None] MFA Rules not processed for user `722c3c0f10694dcf99911d25bd6164a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:33.734271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8ebff31c-cb65-4abf-9cf7-1b4478b0eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:33.763134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:33.815275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:33.855977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80061b0f-601f-41a2-aeb0-f543b635a205 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:33.857963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80061b0f-601f-41a2-aeb0-f543b635a205 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:33.875963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1084/2141] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:33 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:33.908919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1058/2142] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:33 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:33.915057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80061b0f-601f-41a2-aeb0-f543b635a205 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.039213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80061b0f-601f-41a2-aeb0-f543b635a205 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.046253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f9fd8-fba8-41f5-89ca-c5ca793a99ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.047653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f9fd8-fba8-41f5-89ca-c5ca793a99ee admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.048606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f9fd8-fba8-41f5-89ca-c5ca793a99ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.049077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410f9fd8-fba8-41f5-89ca-c5ca793a99ee admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.050320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f9fd8-fba8-41f5-89ca-c5ca793a99ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.053025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410f9fd8-fba8-41f5-89ca-c5ca793a99ee admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.090274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebff31c-cb65-4abf-9cf7-1b4478b0eacb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.092254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8ebff31c-cb65-4abf-9cf7-1b4478b0eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:34.109801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebff31c-cb65-4abf-9cf7-1b4478b0eacb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.113705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1085/2143] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:33 2022] POST /identity/v3/projects => generated 367 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:34.114539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.115253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.115530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.115806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] PATH_INFO: `/v3/projects/fca4d40662594a3aaa112f030c9bc3b6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.117017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fca4d40662594a3aaa112f030c9bc3b6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.117576 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f9fd8-fba8-41f5-89ca-c5ca793a99ee admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.119011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.177960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fca4d40662594a3aaa112f030c9bc3b6', 'request_id': 'req-8dfa829e-7429-4885-b465-143d44455d6c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:34.184064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410f9fd8-fba8-41f5-89ca-c5ca793a99ee admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.185357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fca4d40662594a3aaa112f030c9bc3b6', 'request_id': 'req-8dfa829e-7429-4885-b465-143d44455d6c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:34.191372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fca4d40662594a3aaa112f030c9bc3b6', 'request_id': 'req-8dfa829e-7429-4885-b465-143d44455d6c'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:34.207706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] The token cache is being invalidate because project fca4d40662594a3aaa112f030c9bc3b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:34.208124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:34.208834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8dfa829e-7429-4885-b465-143d44455d6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:34.210728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1059/2144] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:34 2022] DELETE /identity/v3/projects/fca4d40662594a3aaa112f030c9bc3b6 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:34.215557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce070154-b889-4fdd-9a66-8688cae6a2ac None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.215831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce070154-b889-4fdd-9a66-8688cae6a2ac None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.216109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce070154-b889-4fdd-9a66-8688cae6a2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.240946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ce070154-b889-4fdd-9a66-8688cae6a2ac None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:34.244521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ce070154-b889-4fdd-9a66-8688cae6a2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:34.250529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa7658-4257-4ccd-befc-956a7376b42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.252237 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa7658-4257-4ccd-befc-956a7376b42c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.252682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa7658-4257-4ccd-befc-956a7376b42c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.252963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fa7658-4257-4ccd-befc-956a7376b42c admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.254126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa7658-4257-4ccd-befc-956a7376b42c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.256510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fa7658-4257-4ccd-befc-956a7376b42c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.292713 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1086/2145] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:34 2022] POST /identity/v3/users => generated 290 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:34.297065 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa7658-4257-4ccd-befc-956a7376b42c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.300525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3fa7658-4257-4ccd-befc-956a7376b42c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:34.318713 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fa7658-4257-4ccd-befc-956a7376b42c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.324232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba524696-5fc4-4017-af6e-6de813aad78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.324904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba524696-5fc4-4017-af6e-6de813aad78e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.325194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba524696-5fc4-4017-af6e-6de813aad78e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.325656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba524696-5fc4-4017-af6e-6de813aad78e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.326351 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba524696-5fc4-4017-af6e-6de813aad78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.327593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1060/2146] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:34.328087 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba524696-5fc4-4017-af6e-6de813aad78e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.329929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:34.336190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc88f44-6016-41ba-b80d-fc9aad3a18a4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.336657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc88f44-6016-41ba-b80d-fc9aad3a18a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.336790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc88f44-6016-41ba-b80d-fc9aad3a18a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.346825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1087/2147] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:34.348062 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba524696-5fc4-4017-af6e-6de813aad78e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.364608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-efc88f44-6016-41ba-b80d-fc9aad3a18a4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:34.365146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba524696-5fc4-4017-af6e-6de813aad78e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.367560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-efc88f44-6016-41ba-b80d-fc9aad3a18a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:34.371346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184a46d-7cc6-4c0f-be39-98409b54c485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.372100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184a46d-7cc6-4c0f-be39-98409b54c485 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.372434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184a46d-7cc6-4c0f-be39-98409b54c485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.372646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a184a46d-7cc6-4c0f-be39-98409b54c485 admin admin] PATH_INFO: `/v3/projects/43736a60a6854019972fdfea348b19fb/users/ba527ac8abc64baba04075040e7b8dd7/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.383402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1061/2148] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:34.386781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc88f44-6016-41ba-b80d-fc9aad3a18a4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.396864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc88f44-6016-41ba-b80d-fc9aad3a18a4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.402903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184a46d-7cc6-4c0f-be39-98409b54c485 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43736a60a6854019972fdfea348b19fb, user_id=ba527ac8abc64baba04075040e7b8dd7, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.403188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1731f-25a0-4a3d-970c-1df79d36170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.403872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1731f-25a0-4a3d-970c-1df79d36170d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.404227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1731f-25a0-4a3d-970c-1df79d36170d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.404562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a1731f-25a0-4a3d-970c-1df79d36170d admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.405294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1731f-25a0-4a3d-970c-1df79d36170d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:34.405806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1731f-25a0-4a3d-970c-1df79d36170d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.407506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a184a46d-7cc6-4c0f-be39-98409b54c485 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.407676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a1731f-25a0-4a3d-970c-1df79d36170d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.435559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1088/2149] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:34 2022] PUT /identity/v3/projects/43736a60a6854019972fdfea348b19fb/users/ba527ac8abc64baba04075040e7b8dd7/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:34.436747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1062/2150] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:02:34.441528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184a46d-7cc6-4c0f-be39-98409b54c485 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.444527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1731f-25a0-4a3d-970c-1df79d36170d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.467185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a1731f-25a0-4a3d-970c-1df79d36170d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.503526 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:34.553154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a184a46d-7cc6-4c0f-be39-98409b54c485 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.559316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef3949-817f-4cce-8d0b-037d063cad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.560214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef3949-817f-4cce-8d0b-037d063cad9a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.560597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef3949-817f-4cce-8d0b-037d063cad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.560972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ef3949-817f-4cce-8d0b-037d063cad9a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.561725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef3949-817f-4cce-8d0b-037d063cad9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.563585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ef3949-817f-4cce-8d0b-037d063cad9a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.569391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d37778-d761-4c84-abfa-cfe916fa169d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.570306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d37778-d761-4c84-abfa-cfe916fa169d admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.570482 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d37778-d761-4c84-abfa-cfe916fa169d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.570828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d37778-d761-4c84-abfa-cfe916fa169d admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.571541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d37778-d761-4c84-abfa-cfe916fa169d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.573684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d37778-d761-4c84-abfa-cfe916fa169d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.575659 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1089/2151] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:34.580016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef3949-817f-4cce-8d0b-037d063cad9a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.590475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ef3949-817f-4cce-8d0b-037d063cad9a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.597214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ec629-ed8a-4fed-9ed1-130730107a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.598131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ec629-ed8a-4fed-9ed1-130730107a31 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.598557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ec629-ed8a-4fed-9ed1-130730107a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.599357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ec629-ed8a-4fed-9ed1-130730107a31 admin admin] PATH_INFO: `/v3/projects/43736a60a6854019972fdfea348b19fb/users/ba527ac8abc64baba04075040e7b8dd7/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.601977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ec629-ed8a-4fed-9ed1-130730107a31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43736a60a6854019972fdfea348b19fb, user_id=ba527ac8abc64baba04075040e7b8dd7, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.606148 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ec629-ed8a-4fed-9ed1-130730107a31 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.632169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1090/2152] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:34 2022] PUT /identity/v3/projects/43736a60a6854019972fdfea348b19fb/users/ba527ac8abc64baba04075040e7b8dd7/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:34.632452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1063/2153] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:34 2022] POST /identity/v3/projects => generated 383 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:34.632555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ec629-ed8a-4fed-9ed1-130730107a31 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.634534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36495ebd-65e3-42cf-94cb-1516d768e29a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.635145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36495ebd-65e3-42cf-94cb-1516d768e29a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.635589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36495ebd-65e3-42cf-94cb-1516d768e29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.679836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-36495ebd-65e3-42cf-94cb-1516d768e29a None None] MFA Rules not processed for user `ba527ac8abc64baba04075040e7b8dd7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:34.681685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-36495ebd-65e3-42cf-94cb-1516d768e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:34.712131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ec629-ed8a-4fed-9ed1-130730107a31 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.783403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1064/2154] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:34 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:34.794811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6873d-9a5f-48cf-8379-d91e6898a570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.795764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6873d-9a5f-48cf-8379-d91e6898a570 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.796230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36495ebd-65e3-42cf-94cb-1516d768e29a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.796611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6873d-9a5f-48cf-8379-d91e6898a570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.797246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d6873d-9a5f-48cf-8379-d91e6898a570 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.797895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-36495ebd-65e3-42cf-94cb-1516d768e29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:34.798575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6873d-9a5f-48cf-8379-d91e6898a570 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.801734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d6873d-9a5f-48cf-8379-d91e6898a570 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.815007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:34.837166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1091/2155] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:34 2022] POST /identity/v3/users => generated 292 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:34.838390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6873d-9a5f-48cf-8379-d91e6898a570 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.851473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d6873d-9a5f-48cf-8379-d91e6898a570 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.856441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66103dd4-c639-4396-ad48-7ca3ef4a4b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.857057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66103dd4-c639-4396-ad48-7ca3ef4a4b00 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.857339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66103dd4-c639-4396-ad48-7ca3ef4a4b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.857601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66103dd4-c639-4396-ad48-7ca3ef4a4b00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.858351 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66103dd4-c639-4396-ad48-7ca3ef4a4b00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.860537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66103dd4-c639-4396-ad48-7ca3ef4a4b00 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.872271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1092/2156] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:34.875454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36495ebd-65e3-42cf-94cb-1516d768e29a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.876889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66103dd4-c639-4396-ad48-7ca3ef4a4b00 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.881316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe9728-0859-42f2-a237-bb13c567dd0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.882456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe9728-0859-42f2-a237-bb13c567dd0e service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.882928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe9728-0859-42f2-a237-bb13c567dd0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.884043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fe9728-0859-42f2-a237-bb13c567dd0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.887521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66103dd4-c639-4396-ad48-7ca3ef4a4b00 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.892613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c7ae9-ba32-4055-85ee-b0873de5ebf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.893251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c7ae9-ba32-4055-85ee-b0873de5ebf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.893526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c7ae9-ba32-4055-85ee-b0873de5ebf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.893798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20c7ae9-ba32-4055-85ee-b0873de5ebf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.894477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c7ae9-ba32-4055-85ee-b0873de5ebf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.896213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20c7ae9-ba32-4055-85ee-b0873de5ebf0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.897094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe9728-0859-42f2-a237-bb13c567dd0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.898788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fe9728-0859-42f2-a237-bb13c567dd0e service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.908873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1065/2158] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:34 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:02:34.909036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1093/2158] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:34 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:34.913036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe9728-0859-42f2-a237-bb13c567dd0e service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.935694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fe9728-0859-42f2-a237-bb13c567dd0e service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.950455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a3eb8e-9850-426d-b311-988d08c75ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.952195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3eb8e-9850-426d-b311-988d08c75ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.952485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3eb8e-9850-426d-b311-988d08c75ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.952762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a3eb8e-9850-426d-b311-988d08c75ec9 admin admin] PATH_INFO: `/v3/projects/c5c726d5677c46e5b4fdbdf246d74c46/users/f9295a3715d947e1a08b8284829e5918/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.969635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c7ae9-ba32-4055-85ee-b0873de5ebf0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:34.972478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d20c7ae9-ba32-4055-85ee-b0873de5ebf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:34.973092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3eb8e-9850-426d-b311-988d08c75ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5c726d5677c46e5b4fdbdf246d74c46, user_id=f9295a3715d947e1a08b8284829e5918, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:34.985038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a3eb8e-9850-426d-b311-988d08c75ec9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:34.991546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20c7ae9-ba32-4055-85ee-b0873de5ebf0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:34.995986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cad22-6fb3-4371-a864-9b2ed68c6fab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:34.997320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cad22-6fb3-4371-a864-9b2ed68c6fab service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:34.997320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cad22-6fb3-4371-a864-9b2ed68c6fab service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:34.997320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94cad22-6fb3-4371-a864-9b2ed68c6fab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:34.999876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f94cad22-6fb3-4371-a864-9b2ed68c6fab service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:35.009769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1066/2159] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:34 2022] PUT /identity/v3/projects/c5c726d5677c46e5b4fdbdf246d74c46/users/f9295a3715d947e1a08b8284829e5918/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:35.020870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf79a02-0e05-453f-bd97-efae2898e3e2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:35.021143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf79a02-0e05-453f-bd97-efae2898e3e2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:35.022143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf79a02-0e05-453f-bd97-efae2898e3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:35.047673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:35.060948 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2bf79a02-0e05-453f-bd97-efae2898e3e2 None None] MFA Rules not processed for user `f9295a3715d947e1a08b8284829e5918`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:35.063437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2bf79a02-0e05-453f-bd97-efae2898e3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:35.097745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cad22-6fb3-4371-a864-9b2ed68c6fab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:35.099182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94cad22-6fb3-4371-a864-9b2ed68c6fab service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:35.117560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1094/2160] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:35.177545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1067/2161] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:35.179080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cad22-6fb3-4371-a864-9b2ed68c6fab service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:35.192294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94cad22-6fb3-4371-a864-9b2ed68c6fab service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:35.263442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde4e24-67d5-4b9d-ba99-1b1f5d0a4eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:35.264082 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde4e24-67d5-4b9d-ba99-1b1f5d0a4eb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:35.264364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde4e24-67d5-4b9d-ba99-1b1f5d0a4eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:35.264647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdde4e24-67d5-4b9d-ba99-1b1f5d0a4eb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:35.265301 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde4e24-67d5-4b9d-ba99-1b1f5d0a4eb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:35.267280 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdde4e24-67d5-4b9d-ba99-1b1f5d0a4eb6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:35.312908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1095/2162] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:35 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:35.317148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf79a02-0e05-453f-bd97-efae2898e3e2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:35.332991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf79a02-0e05-453f-bd97-efae2898e3e2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:35.412812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8eaf6-792d-4187-904c-1bfc9f953ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:35.413471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8eaf6-792d-4187-904c-1bfc9f953ae4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:35.414154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8eaf6-792d-4187-904c-1bfc9f953ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:35.414436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8eaf6-792d-4187-904c-1bfc9f953ae4 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:35.415080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8eaf6-792d-4187-904c-1bfc9f953ae4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:35.418706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8eaf6-792d-4187-904c-1bfc9f953ae4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:35.450816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1068/2163] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:35 2022] POST /identity/v3/users => generated 293 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:35.458347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde4e24-67d5-4b9d-ba99-1b1f5d0a4eb6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:35.482741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdde4e24-67d5-4b9d-ba99-1b1f5d0a4eb6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:35.498559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccf467-d4f5-4e98-b00f-46dd63ad33e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:35.499198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccf467-d4f5-4e98-b00f-46dd63ad33e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:35.499475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccf467-d4f5-4e98-b00f-46dd63ad33e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:35.499745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fccf467-d4f5-4e98-b00f-46dd63ad33e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:35.500445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccf467-d4f5-4e98-b00f-46dd63ad33e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:35.504356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fccf467-d4f5-4e98-b00f-46dd63ad33e8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:35.537360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8eaf6-792d-4187-904c-1bfc9f953ae4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:35.541354 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1096/2164] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:35.555014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8eaf6-792d-4187-904c-1bfc9f953ae4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:35.563951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c3f6bb2-0652-4051-8e61-7fa23d8495b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:35.564687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f6bb2-0652-4051-8e61-7fa23d8495b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:35.565028 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f6bb2-0652-4051-8e61-7fa23d8495b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:35.565365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c3f6bb2-0652-4051-8e61-7fa23d8495b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:35.566121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f6bb2-0652-4051-8e61-7fa23d8495b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:35.569681 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c3f6bb2-0652-4051-8e61-7fa23d8495b9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:35.587357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1069/2165] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:35.614986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccf467-d4f5-4e98-b00f-46dd63ad33e8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:35.632182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fccf467-d4f5-4e98-b00f-46dd63ad33e8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:35.640412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9fdfb-6020-4c01-874f-b9b6a0cee948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:35.641103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9fdfb-6020-4c01-874f-b9b6a0cee948 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:35.641378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9fdfb-6020-4c01-874f-b9b6a0cee948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:35.642312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d9fdfb-6020-4c01-874f-b9b6a0cee948 admin admin] PATH_INFO: `/v3/projects/1af8464269bc492e9ad1c2ff812e0eea/users/5fb05841214a46f8954f12472b866bb7/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:35.662476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9fdfb-6020-4c01-874f-b9b6a0cee948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1af8464269bc492e9ad1c2ff812e0eea, user_id=5fb05841214a46f8954f12472b866bb7, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:35.665286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d9fdfb-6020-4c01-874f-b9b6a0cee948 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:35.683509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1097/2166] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:35 2022] PUT /identity/v3/projects/1af8464269bc492e9ad1c2ff812e0eea/users/5fb05841214a46f8954f12472b866bb7/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:35.696049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cefdf-68d8-41f0-842a-c2e8077ac228 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:35.696403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cefdf-68d8-41f0-842a-c2e8077ac228 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:35.696745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2cefdf-68d8-41f0-842a-c2e8077ac228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:35.729447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-4c2cefdf-68d8-41f0-842a-c2e8077ac228 None None] MFA Rules not processed for user `5fb05841214a46f8954f12472b866bb7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:35.731045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4c2cefdf-68d8-41f0-842a-c2e8077ac228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:35.801549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1070/2167] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:35 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:35.806748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9fdfb-6020-4c01-874f-b9b6a0cee948 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:35.827722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:35.886702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d9fdfb-6020-4c01-874f-b9b6a0cee948 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:35.891814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460cfe2-7363-4bc9-9e89-b39e73ded084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:35.892429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460cfe2-7363-4bc9-9e89-b39e73ded084 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:35.892705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460cfe2-7363-4bc9-9e89-b39e73ded084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:35.892970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7460cfe2-7363-4bc9-9e89-b39e73ded084 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:35.893647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460cfe2-7363-4bc9-9e89-b39e73ded084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:35.896196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7460cfe2-7363-4bc9-9e89-b39e73ded084 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:35.941390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1098/2168] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:35 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:35.945839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cefdf-68d8-41f0-842a-c2e8077ac228 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:35.957462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2cefdf-68d8-41f0-842a-c2e8077ac228 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:35.961396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:36.020557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c0d81-b630-4229-92e8-2e2d42bd0356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:36.021180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c0d81-b630-4229-92e8-2e2d42bd0356 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:36.021455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c0d81-b630-4229-92e8-2e2d42bd0356 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:36.021724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c0d81-b630-4229-92e8-2e2d42bd0356 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:36.022434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c0d81-b630-4229-92e8-2e2d42bd0356 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:36.024359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c0d81-b630-4229-92e8-2e2d42bd0356 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:36.048094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1071/2169] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:35 2022] POST /identity/v3/users => generated 299 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:36.052499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460cfe2-7363-4bc9-9e89-b39e73ded084 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:36.064158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7460cfe2-7363-4bc9-9e89-b39e73ded084 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:36.069041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f8ccd-f04a-47c5-b7cd-12511fd0569e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:36.069759 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f8ccd-f04a-47c5-b7cd-12511fd0569e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:36.070107 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f8ccd-f04a-47c5-b7cd-12511fd0569e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:36.070425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f8ccd-f04a-47c5-b7cd-12511fd0569e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:36.071150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f8ccd-f04a-47c5-b7cd-12511fd0569e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:36.073068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f8ccd-f04a-47c5-b7cd-12511fd0569e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:36.086260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1099/2170] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:36.090795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c0d81-b630-4229-92e8-2e2d42bd0356 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:36.101342 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c0d81-b630-4229-92e8-2e2d42bd0356 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:36.107333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6766a4-e445-442b-b028-d2956b2b52a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:36.108038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6766a4-e445-442b-b028-d2956b2b52a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:36.108318 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6766a4-e445-442b-b028-d2956b2b52a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:36.108593 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6766a4-e445-442b-b028-d2956b2b52a5 admin admin] PATH_INFO: `/v3/projects/0ccc330bca994055a085fb0c9e6679ac/users/5b28b21dede045f980246eb11aa47937/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:36.128176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6766a4-e445-442b-b028-d2956b2b52a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccc330bca994055a085fb0c9e6679ac, user_id=5b28b21dede045f980246eb11aa47937, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:36.130917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6766a4-e445-442b-b028-d2956b2b52a5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:36.146013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1072/2171] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:36 2022] PUT /identity/v3/projects/0ccc330bca994055a085fb0c9e6679ac/users/5b28b21dede045f980246eb11aa47937/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:36.149850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f8ccd-f04a-47c5-b7cd-12511fd0569e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:36.160206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f8ccd-f04a-47c5-b7cd-12511fd0569e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:36.226962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49313ed3-e6d1-4a9b-a5cb-7f2b739b4537 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:36.227654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313ed3-e6d1-4a9b-a5cb-7f2b739b4537 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:36.227940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313ed3-e6d1-4a9b-a5cb-7f2b739b4537 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:36.228192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49313ed3-e6d1-4a9b-a5cb-7f2b739b4537 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:36.228863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49313ed3-e6d1-4a9b-a5cb-7f2b739b4537 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:36.230816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49313ed3-e6d1-4a9b-a5cb-7f2b739b4537 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:36.243194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1100/2172] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:36.247635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6766a4-e445-442b-b028-d2956b2b52a5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:36.258845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6766a4-e445-442b-b028-d2956b2b52a5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:36.264736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00756f9c-6b3c-4494-a557-ee49911770e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:36.265427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00756f9c-6b3c-4494-a557-ee49911770e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:36.265709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00756f9c-6b3c-4494-a557-ee49911770e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:36.265997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00756f9c-6b3c-4494-a557-ee49911770e7 admin admin] PATH_INFO: `/v3/projects/0ccc330bca994055a085fb0c9e6679ac/users/5b28b21dede045f980246eb11aa47937/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:36.268322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00756f9c-6b3c-4494-a557-ee49911770e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccc330bca994055a085fb0c9e6679ac, user_id=5b28b21dede045f980246eb11aa47937, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:36.273086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00756f9c-6b3c-4494-a557-ee49911770e7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:36.286697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1073/2173] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:36 2022] PUT /identity/v3/projects/0ccc330bca994055a085fb0c9e6679ac/users/5b28b21dede045f980246eb11aa47937/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:36.297738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355ae71-f22e-403a-949e-c749c1df66ec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:36.298291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355ae71-f22e-403a-949e-c749c1df66ec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:36.298570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3355ae71-f22e-403a-949e-c749c1df66ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:36.335088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3355ae71-f22e-403a-949e-c749c1df66ec None None] MFA Rules not processed for user `5b28b21dede045f980246eb11aa47937`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:36.336603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3355ae71-f22e-403a-949e-c749c1df66ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:36.409440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1101/2174] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:36 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:36.421325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00756f9c-6b3c-4494-a557-ee49911770e7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:36.481326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00756f9c-6b3c-4494-a557-ee49911770e7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:36.485191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd636bd-e769-4a21-baca-83896d8893ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:36.485813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd636bd-e769-4a21-baca-83896d8893ef service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:36.486136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd636bd-e769-4a21-baca-83896d8893ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:36.486392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd636bd-e769-4a21-baca-83896d8893ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:36.545801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd636bd-e769-4a21-baca-83896d8893ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:36.547351 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd636bd-e769-4a21-baca-83896d8893ef service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:36.557328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1074/2175] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:36 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:37.140691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355ae71-f22e-403a-949e-c749c1df66ec None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:37.142224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3355ae71-f22e-403a-949e-c749c1df66ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:37.161510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:37.232230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3355ae71-f22e-403a-949e-c749c1df66ec None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:37.236838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:37.237571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:37.237851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:37.238224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] PATH_INFO: `/v3/users/292b0d583c4043028038e501df1793cf` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:37.239249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] RBAC: Authorizing `identity:delete_user(user_id=292b0d583c4043028038e501df1793cf)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:37.241175 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:37.297020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '292b0d583c4043028038e501df1793cf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.297826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '292b0d583c4043028038e501df1793cf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.310216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '292b0d583c4043028038e501df1793cf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.326352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '292b0d583c4043028038e501df1793cf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.327097 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '292b0d583c4043028038e501df1793cf'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.340355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1102/2176] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:37 2022] DELETE /identity/v3/users/292b0d583c4043028038e501df1793cf => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:37.345101 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222baca-c5a8-4649-bb8b-bbf7d6147107 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:37.345930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222baca-c5a8-4649-bb8b-bbf7d6147107 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:37.346247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1222baca-c5a8-4649-bb8b-bbf7d6147107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:37.366506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1222baca-c5a8-4649-bb8b-bbf7d6147107 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:37.367710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1222baca-c5a8-4649-bb8b-bbf7d6147107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:37.376736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:37.439578 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1075/2177] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:37.450069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:37.455197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:37.516571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210f68b-b1be-4add-8a29-30ac1e7ae498 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:37.520557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291f84d-6423-4f43-960f-2b3b17da365f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:37.521169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291f84d-6423-4f43-960f-2b3b17da365f service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:37.521452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291f84d-6423-4f43-960f-2b3b17da365f service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:37.521726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291f84d-6423-4f43-960f-2b3b17da365f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:37.531478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291f84d-6423-4f43-960f-2b3b17da365f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:37.532918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291f84d-6423-4f43-960f-2b3b17da365f service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:37.542949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1103/2178] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:37.829011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222baca-c5a8-4649-bb8b-bbf7d6147107 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:37.830727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1222baca-c5a8-4649-bb8b-bbf7d6147107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:37.843741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1222baca-c5a8-4649-bb8b-bbf7d6147107 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:37.849846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:37.850661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:37.851012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:37.851348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] PATH_INFO: `/v3/projects/ac5e4f5e9f9e49f1b3ab45092a6e3013` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:37.852823 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac5e4f5e9f9e49f1b3ab45092a6e3013)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:37.854897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:37.906672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac5e4f5e9f9e49f1b3ab45092a6e3013', 'request_id': 'req-6321f6c0-c2cc-4df7-9362-530df73a168a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.913630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac5e4f5e9f9e49f1b3ab45092a6e3013', 'request_id': 'req-6321f6c0-c2cc-4df7-9362-530df73a168a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.919978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac5e4f5e9f9e49f1b3ab45092a6e3013', 'request_id': 'req-6321f6c0-c2cc-4df7-9362-530df73a168a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.933355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] The token cache is being invalidate because project ac5e4f5e9f9e49f1b3ab45092a6e3013 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:37.933650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.934333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6321f6c0-c2cc-4df7-9362-530df73a168a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:37.935879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1076/2179] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:37 2022] DELETE /identity/v3/projects/ac5e4f5e9f9e49f1b3ab45092a6e3013 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:37.941106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8f140-5f84-4b71-85bf-c926cbff574b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:37.941378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8f140-5f84-4b71-85bf-c926cbff574b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:37.941676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8f140-5f84-4b71-85bf-c926cbff574b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:37.961979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-86a8f140-5f84-4b71-85bf-c926cbff574b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:37.963276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-86a8f140-5f84-4b71-85bf-c926cbff574b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:38.000401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:38.044635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1104/2180] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:38.052860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9062c7-4ae0-496f-8460-4a23d69ac5e1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.053364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9062c7-4ae0-496f-8460-4a23d69ac5e1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.053740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9062c7-4ae0-496f-8460-4a23d69ac5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.086475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9e9062c7-4ae0-496f-8460-4a23d69ac5e1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:38.088083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9e9062c7-4ae0-496f-8460-4a23d69ac5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:38.102720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1077/2181] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:38.104233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8f140-5f84-4b71-85bf-c926cbff574b None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.129998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8f140-5f84-4b71-85bf-c926cbff574b None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:38.143610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:38.144684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.145104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.145369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.146222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:38.146782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:38.148854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:38.168962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1105/2182] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:38.173312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9062c7-4ae0-496f-8460-4a23d69ac5e1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.190155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9062c7-4ae0-496f-8460-4a23d69ac5e1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:38.203327 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.205079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:38.206243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a32dd-0127-4039-a914-95ca48c32388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:38.206979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a32dd-0127-4039-a914-95ca48c32388 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.207360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a32dd-0127-4039-a914-95ca48c32388 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.207737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a32dd-0127-4039-a914-95ca48c32388 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.208492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a32dd-0127-4039-a914-95ca48c32388 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:38.210563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a32dd-0127-4039-a914-95ca48c32388 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:38.264509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1078/2183] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:38 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:38.265386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a32dd-0127-4039-a914-95ca48c32388 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.300679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c534ada-8dfb-4232-9257-54ade4abd80e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:38.353858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a32dd-0127-4039-a914-95ca48c32388 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:38.367938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5525793-d551-498e-9c08-5f10ee3c2dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:38.368806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5525793-d551-498e-9c08-5f10ee3c2dd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.369181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5525793-d551-498e-9c08-5f10ee3c2dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.369552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5525793-d551-498e-9c08-5f10ee3c2dd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.370782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6567ce-eb81-4590-ba5a-e7a62f04ffd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:38.371610 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5525793-d551-498e-9c08-5f10ee3c2dd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:38.373932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5525793-d551-498e-9c08-5f10ee3c2dd4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:38.374837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6567ce-eb81-4590-ba5a-e7a62f04ffd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.375075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6567ce-eb81-4590-ba5a-e7a62f04ffd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.375629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6567ce-eb81-4590-ba5a-e7a62f04ffd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.379819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2a6567ce-eb81-4590-ba5a-e7a62f04ffd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:38.396452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:38.409885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1079/2184] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:38 2022] POST /identity/v3/users => generated 287 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:38.414320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5525793-d551-498e-9c08-5f10ee3c2dd4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.428439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5525793-d551-498e-9c08-5f10ee3c2dd4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:38.440329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dfa43-ee9d-4471-9f79-55cf1078eea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:38.442161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dfa43-ee9d-4471-9f79-55cf1078eea1 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.442666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dfa43-ee9d-4471-9f79-55cf1078eea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.443122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dfa43-ee9d-4471-9f79-55cf1078eea1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.443988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dfa43-ee9d-4471-9f79-55cf1078eea1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:38.445865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7dfa43-ee9d-4471-9f79-55cf1078eea1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:38.459179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1080/2185] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:38.460356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dfa43-ee9d-4471-9f79-55cf1078eea1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.462357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7a7dfa43-ee9d-4471-9f79-55cf1078eea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:38.487960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7dfa43-ee9d-4471-9f79-55cf1078eea1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:38.493585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6567ce-eb81-4590-ba5a-e7a62f04ffd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:38.495189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6567ce-eb81-4590-ba5a-e7a62f04ffd8 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:38.497346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:38.498139 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.498481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.498838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] PATH_INFO: `/v3/users/dd8e8bd1582145608a8da684a103c94e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.500186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd8e8bd1582145608a8da684a103c94e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:38.502408 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:38.529898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1106/2186] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:38 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:02:38.533110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6567ce-eb81-4590-ba5a-e7a62f04ffd8 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.552701 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6567ce-eb81-4590-ba5a-e7a62f04ffd8 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:38.571734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809f969-868f-4606-b293-6f757b1ee8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:38.572611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809f969-868f-4606-b293-6f757b1ee8fe admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.573103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809f969-868f-4606-b293-6f757b1ee8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.573536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c809f969-868f-4606-b293-6f757b1ee8fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.574218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809f969-868f-4606-b293-6f757b1ee8fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:38.574335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd8e8bd1582145608a8da684a103c94e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:38.575980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c809f969-868f-4606-b293-6f757b1ee8fe admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:38.593515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd8e8bd1582145608a8da684a103c94e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:38.595302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1107/2187] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:38 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:38.599713 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809f969-868f-4606-b293-6f757b1ee8fe admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.611287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c809f969-868f-4606-b293-6f757b1ee8fe admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:38.616213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9712b7-f566-40d6-94df-b51048bd6d9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:38.616910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9712b7-f566-40d6-94df-b51048bd6d9d admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.617193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9712b7-f566-40d6-94df-b51048bd6d9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.617467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9712b7-f566-40d6-94df-b51048bd6d9d admin admin] PATH_INFO: `/v3/projects/18e996c5a0db410999d4b50a04dc626a/users/e32889b846f1431ca5169ee1c9d524db/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.630315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd8e8bd1582145608a8da684a103c94e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:38.631093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd8e8bd1582145608a8da684a103c94e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:38.639777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-848d450d-8e05-4f4a-9878-5082a08556ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd8e8bd1582145608a8da684a103c94e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:38.649454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1081/2188] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:38 2022] DELETE /identity/v3/users/dd8e8bd1582145608a8da684a103c94e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:38.651502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862facb-119d-45f5-9079-bc489e72d0fe None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.652066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862facb-119d-45f5-9079-bc489e72d0fe None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.652300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6862facb-119d-45f5-9079-bc489e72d0fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.662882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9712b7-f566-40d6-94df-b51048bd6d9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=18e996c5a0db410999d4b50a04dc626a, user_id=e32889b846f1431ca5169ee1c9d524db, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:38.671556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9712b7-f566-40d6-94df-b51048bd6d9d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:38.685078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6862facb-119d-45f5-9079-bc489e72d0fe None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:38.686608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6862facb-119d-45f5-9079-bc489e72d0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:38.692973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:38.699478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1108/2189] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:38 2022] PUT /identity/v3/projects/18e996c5a0db410999d4b50a04dc626a/users/e32889b846f1431ca5169ee1c9d524db/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:38.711369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957848c8-6670-4c0d-8345-a1eb22e6e48d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:38.711765 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957848c8-6670-4c0d-8345-a1eb22e6e48d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:38.712165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957848c8-6670-4c0d-8345-a1eb22e6e48d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:38.753158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-957848c8-6670-4c0d-8345-a1eb22e6e48d None None] MFA Rules not processed for user `e32889b846f1431ca5169ee1c9d524db`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:38.754930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-957848c8-6670-4c0d-8345-a1eb22e6e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:38.818198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1082/2190] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:38.829576 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862facb-119d-45f5-9079-bc489e72d0fe None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.831339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6862facb-119d-45f5-9079-bc489e72d0fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:38.859114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1109/2191] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:38.908861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957848c8-6670-4c0d-8345-a1eb22e6e48d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:38.910886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-957848c8-6670-4c0d-8345-a1eb22e6e48d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:38.946819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6862facb-119d-45f5-9079-bc489e72d0fe None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:38.960687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:39.017977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dce6b-ac34-4b1b-bde7-2273218ad27e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:39.019303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dce6b-ac34-4b1b-bde7-2273218ad27e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:39.020371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dce6b-ac34-4b1b-bde7-2273218ad27e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:39.020759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dce6b-ac34-4b1b-bde7-2273218ad27e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:39.053039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957848c8-6670-4c0d-8345-a1eb22e6e48d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:39.062378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679aead-d206-4012-ae87-0a2e7232ef3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:39.064686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679aead-d206-4012-ae87-0a2e7232ef3c service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:39.066573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679aead-d206-4012-ae87-0a2e7232ef3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:39.066885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4679aead-d206-4012-ae87-0a2e7232ef3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:39.076666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679aead-d206-4012-ae87-0a2e7232ef3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:39.079254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4679aead-d206-4012-ae87-0a2e7232ef3c service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:39.089928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1110/2192] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:02:39.121964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679aead-d206-4012-ae87-0a2e7232ef3c service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:39.126161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4679aead-d206-4012-ae87-0a2e7232ef3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:39.137587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dce6b-ac34-4b1b-bde7-2273218ad27e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:39.140214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dce6b-ac34-4b1b-bde7-2273218ad27e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:39.162461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1083/2193] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:02:39.165535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4679aead-d206-4012-ae87-0a2e7232ef3c service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:39.174901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:39.175711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:39.175995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:39.176274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] PATH_INFO: `/v3/users/e32889b846f1431ca5169ee1c9d524db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:39.177616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32889b846f1431ca5169ee1c9d524db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:39.180125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:39.262068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32889b846f1431ca5169ee1c9d524db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.264488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32889b846f1431ca5169ee1c9d524db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.280544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32889b846f1431ca5169ee1c9d524db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.306359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32889b846f1431ca5169ee1c9d524db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.308132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32889b846f1431ca5169ee1c9d524db'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.328324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1111/2194] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:39 2022] DELETE /identity/v3/users/e32889b846f1431ca5169ee1c9d524db => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:39.333586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3978-b2ea-4e39-b292-2c37c3eba168 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:39.334159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3978-b2ea-4e39-b292-2c37c3eba168 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:39.334660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb3978-b2ea-4e39-b292-2c37c3eba168 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:39.361364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-55bb3978-b2ea-4e39-b292-2c37c3eba168 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:39.362831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-55bb3978-b2ea-4e39-b292-2c37c3eba168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:39.463584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1084/2195] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:39.474400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:39.476587 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:39.518274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:39.576943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3978-b2ea-4e39-b292-2c37c3eba168 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:39.579815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-55bb3978-b2ea-4e39-b292-2c37c3eba168 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:39.643235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd697aa-f8be-4d27-ab1b-f2b20e3006c2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:39.654319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c8f25-e13a-4922-b423-13cdfdc122df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:39.654966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c8f25-e13a-4922-b423-13cdfdc122df service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:39.655248 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c8f25-e13a-4922-b423-13cdfdc122df service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:39.655523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4c8f25-e13a-4922-b423-13cdfdc122df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:39.673912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c8f25-e13a-4922-b423-13cdfdc122df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:39.676179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4c8f25-e13a-4922-b423-13cdfdc122df service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:39.686209 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb3978-b2ea-4e39-b292-2c37c3eba168 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:39.690525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:39.691530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:39.692184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:39.692658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:39.695155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:39.703057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1112/2196] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:39.705699 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c8f25-e13a-4922-b423-13cdfdc122df service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:39.708201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dc4c8f25-e13a-4922-b423-13cdfdc122df service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:39.723258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:39.750669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4c8f25-e13a-4922-b423-13cdfdc122df service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:39.758434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:39.759464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:39.760253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:39.760722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] PATH_INFO: `/v3/users/722c3c0f10694dcf99911d25bd6164a7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:39.762694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=722c3c0f10694dcf99911d25bd6164a7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:39.765154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:39.807325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:39.808861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:39.822186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1085/2197] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:39 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:39.823351 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:39.825152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:39.865180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0094ab-4df7-4ef7-8e32-24f093c65256 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:39.872125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:39.873158 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:39.873668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:39.873781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '722c3c0f10694dcf99911d25bd6164a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.873957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] PATH_INFO: `/v3/projects/496f4307ecfb4214bd705e3a9a8408ed` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:39.874521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '722c3c0f10694dcf99911d25bd6164a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.875447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] RBAC: Authorizing `identity:delete_project(project_id=496f4307ecfb4214bd705e3a9a8408ed)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:39.877908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:39.887375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '722c3c0f10694dcf99911d25bd6164a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.904681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '722c3c0f10694dcf99911d25bd6164a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.905434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b462447-5bbe-436b-9287-7bdf53360076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '722c3c0f10694dcf99911d25bd6164a7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.922217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1113/2198] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:39 2022] DELETE /identity/v3/users/722c3c0f10694dcf99911d25bd6164a7 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:39.927263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff96b9-7ceb-47ec-86c5-9ea499bca0ee None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:39.927658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff96b9-7ceb-47ec-86c5-9ea499bca0ee None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:39.928159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ff96b9-7ceb-47ec-86c5-9ea499bca0ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:39.951607 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a4ff96b9-7ceb-47ec-86c5-9ea499bca0ee None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:39.953532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a4ff96b9-7ceb-47ec-86c5-9ea499bca0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:39.961980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '496f4307ecfb4214bd705e3a9a8408ed', 'request_id': 'req-a9a824fb-b282-4d53-9f05-588e3cff450f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.969128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '496f4307ecfb4214bd705e3a9a8408ed', 'request_id': 'req-a9a824fb-b282-4d53-9f05-588e3cff450f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.976509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '496f4307ecfb4214bd705e3a9a8408ed', 'request_id': 'req-a9a824fb-b282-4d53-9f05-588e3cff450f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.993650 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] The token cache is being invalidate because project 496f4307ecfb4214bd705e3a9a8408ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:39.994016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.995273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:39.997134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1086/2199] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:39 2022] DELETE /identity/v3/projects/496f4307ecfb4214bd705e3a9a8408ed => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:40.001545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.003153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.047812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1114/2200] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:40.061217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff96b9-7ceb-47ec-86c5-9ea499bca0ee None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.062989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a4ff96b9-7ceb-47ec-86c5-9ea499bca0ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.080772 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a824fb-b282-4d53-9f05-588e3cff450f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.086272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:40.086416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.087046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.087356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.087596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] PATH_INFO: `/v3/users/3e776fc3ce3449788986e1282ec1bb6f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.088896 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e776fc3ce3449788986e1282ec1bb6f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.091164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.136166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ff96b9-7ceb-47ec-86c5-9ea499bca0ee None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.143438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba4f80-9edb-452e-bdee-215dca84a535 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.144222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba4f80-9edb-452e-bdee-215dca84a535 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.144622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba4f80-9edb-452e-bdee-215dca84a535 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.144855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba4f80-9edb-452e-bdee-215dca84a535 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.153474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:40.156745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba4f80-9edb-452e-bdee-215dca84a535 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.158960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba4f80-9edb-452e-bdee-215dca84a535 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.160275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e776fc3ce3449788986e1282ec1bb6f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.169988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1115/2201] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:40.170839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba4f80-9edb-452e-bdee-215dca84a535 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.172373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-96ba4f80-9edb-452e-bdee-215dca84a535 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.172911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e776fc3ce3449788986e1282ec1bb6f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.185883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba4f80-9edb-452e-bdee-215dca84a535 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.190747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.191274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e776fc3ce3449788986e1282ec1bb6f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.191389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.191660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.191937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] PATH_INFO: `/v3/projects/18e996c5a0db410999d4b50a04dc626a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.192314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e776fc3ce3449788986e1282ec1bb6f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.193240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18e996c5a0db410999d4b50a04dc626a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.195669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.200714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f02a6670-3ddc-4052-90c1-fa4223c7e0b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e776fc3ce3449788986e1282ec1bb6f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.208627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1087/2202] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:40 2022] DELETE /identity/v3/users/3e776fc3ce3449788986e1282ec1bb6f => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:40.263745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18e996c5a0db410999d4b50a04dc626a', 'request_id': 'req-fe1bc087-7599-4049-812c-da70f5f2c1f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.271804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18e996c5a0db410999d4b50a04dc626a', 'request_id': 'req-fe1bc087-7599-4049-812c-da70f5f2c1f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.279840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18e996c5a0db410999d4b50a04dc626a', 'request_id': 'req-fe1bc087-7599-4049-812c-da70f5f2c1f6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.295645 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] The token cache is being invalidate because project 18e996c5a0db410999d4b50a04dc626a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:40.295946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.296624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-fe1bc087-7599-4049-812c-da70f5f2c1f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.298738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1116/2203] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:40 2022] DELETE /identity/v3/projects/18e996c5a0db410999d4b50a04dc626a => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:40.304248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bc6e2-06fb-4c40-bffc-f69969becc6b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.304677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bc6e2-06fb-4c40-bffc-f69969becc6b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.305080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892bc6e2-06fb-4c40-bffc-f69969becc6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.331287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-892bc6e2-06fb-4c40-bffc-f69969becc6b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:40.332550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-892bc6e2-06fb-4c40-bffc-f69969becc6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.407122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1088/2204] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:40.416551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fd6b4-19d2-4cae-9a39-216b72b05220 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.417371 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fd6b4-19d2-4cae-9a39-216b72b05220 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.417723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326fd6b4-19d2-4cae-9a39-216b72b05220 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.440758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-326fd6b4-19d2-4cae-9a39-216b72b05220 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:40.442260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-326fd6b4-19d2-4cae-9a39-216b72b05220 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.451994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1117/2205] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:40.456262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bc6e2-06fb-4c40-bffc-f69969becc6b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.468003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892bc6e2-06fb-4c40-bffc-f69969becc6b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.473812 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.474774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.474774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.475002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.475623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:40.475910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.477541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.491666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1089/2206] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:02:40.496103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fd6b4-19d2-4cae-9a39-216b72b05220 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.508146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326fd6b4-19d2-4cae-9a39-216b72b05220 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.513262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.516025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.519102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15df72-e6fd-4ade-8822-769be62cda86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.519741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15df72-e6fd-4ade-8822-769be62cda86 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.520024 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15df72-e6fd-4ade-8822-769be62cda86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.520297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca15df72-e6fd-4ade-8822-769be62cda86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.520948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15df72-e6fd-4ade-8822-769be62cda86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.522990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca15df72-e6fd-4ade-8822-769be62cda86 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.536364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb951f62-4f82-4a89-99db-27fe6f8200a5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.542471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.543322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.543769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.544180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] PATH_INFO: `/v3/projects/9d540eb551714dff92b908017ccfcad7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.545484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=9d540eb551714dff92b908017ccfcad7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.547615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.565928 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1118/2207] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:40 2022] POST /identity/v3/projects => generated 383 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:40.566828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15df72-e6fd-4ade-8822-769be62cda86 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.568402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ca15df72-e6fd-4ade-8822-769be62cda86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.616251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9d540eb551714dff92b908017ccfcad7', 'request_id': 'req-3412d733-f97d-45e1-aecf-7d36174b48ed'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.623701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9d540eb551714dff92b908017ccfcad7', 'request_id': 'req-3412d733-f97d-45e1-aecf-7d36174b48ed'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.629646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9d540eb551714dff92b908017ccfcad7', 'request_id': 'req-3412d733-f97d-45e1-aecf-7d36174b48ed'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.639612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca15df72-e6fd-4ade-8822-769be62cda86 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.643509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] The token cache is being invalidate because project 9d540eb551714dff92b908017ccfcad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:40.643798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.644431 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.646105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1090/2208] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:40 2022] DELETE /identity/v3/projects/9d540eb551714dff92b908017ccfcad7 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:40.649291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.649291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.658370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:40.682908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:40.700583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.701236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.701520 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.701791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] PATH_INFO: `/v3/projects/487ce54f65164d799b7f1c21eb8580a1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.702947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] RBAC: Authorizing `identity:delete_project(project_id=487ce54f65164d799b7f1c21eb8580a1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.704867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.717304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3412d733-f97d-45e1-aecf-7d36174b48ed admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.722052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23397c5-775b-4553-aeaa-8f03556c4c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.722701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23397c5-775b-4553-aeaa-8f03556c4c81 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.722941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23397c5-775b-4553-aeaa-8f03556c4c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.723213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23397c5-775b-4553-aeaa-8f03556c4c81 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.723845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23397c5-775b-4553-aeaa-8f03556c4c81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.725790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23397c5-775b-4553-aeaa-8f03556c4c81 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.749003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1091/2209] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:40 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:40.749915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23397c5-775b-4553-aeaa-8f03556c4c81 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.751513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c23397c5-775b-4553-aeaa-8f03556c4c81 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.755546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '487ce54f65164d799b7f1c21eb8580a1', 'request_id': 'req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.762898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '487ce54f65164d799b7f1c21eb8580a1', 'request_id': 'req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.763718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23397c5-775b-4553-aeaa-8f03556c4c81 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.769510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '487ce54f65164d799b7f1c21eb8580a1', 'request_id': 'req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.783711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] The token cache is being invalidate because project 487ce54f65164d799b7f1c21eb8580a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:40.784341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.784684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.786517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1119/2210] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:40 2022] DELETE /identity/v3/projects/487ce54f65164d799b7f1c21eb8580a1 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:40.787081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.788461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.827945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.828605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.828889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.829169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] PATH_INFO: `/v3/users/ba527ac8abc64baba04075040e7b8dd7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.830376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba527ac8abc64baba04075040e7b8dd7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.832334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.876705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a7d56-4d3d-4758-a4d3-b8ff48bcc07f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.881566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c01491-de97-4f5d-943f-98e32e7ba230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.882236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c01491-de97-4f5d-943f-98e32e7ba230 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.882589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c01491-de97-4f5d-943f-98e32e7ba230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.882937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c01491-de97-4f5d-943f-98e32e7ba230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.883727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c01491-de97-4f5d-943f-98e32e7ba230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.887724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c01491-de97-4f5d-943f-98e32e7ba230 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.901126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1120/2211] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:40.902272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c01491-de97-4f5d-943f-98e32e7ba230 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.904407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-13c01491-de97-4f5d-943f-98e32e7ba230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:40.913168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba527ac8abc64baba04075040e7b8dd7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.924870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c01491-de97-4f5d-943f-98e32e7ba230 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:40.928911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba527ac8abc64baba04075040e7b8dd7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.933978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:40.934782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:40.935166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:40.935532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] PATH_INFO: `/v3/users/cc7f09f62ba44400a52e52985f2ed3e0` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:40.939213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc7f09f62ba44400a52e52985f2ed3e0)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:40.942319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:40.960762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba527ac8abc64baba04075040e7b8dd7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.962778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba527ac8abc64baba04075040e7b8dd7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.975414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba527ac8abc64baba04075040e7b8dd7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:40.990372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1092/2212] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:40 2022] DELETE /identity/v3/users/ba527ac8abc64baba04075040e7b8dd7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:40.996588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:40.999147 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.044219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:41.050252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc7f09f62ba44400a52e52985f2ed3e0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.051304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc7f09f62ba44400a52e52985f2ed3e0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.076857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc7f09f62ba44400a52e52985f2ed3e0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.102723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc7f09f62ba44400a52e52985f2ed3e0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.103127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:41.104043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc7f09f62ba44400a52e52985f2ed3e0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.138094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1121/2213] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:40 2022] DELETE /identity/v3/users/cc7f09f62ba44400a52e52985f2ed3e0 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:41.172782 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29c7141-5188-4d79-a158-f2e09f12b932 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:41.310364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea90ea1-4c74-4bee-a6d6-c72b3f180da2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:41.311360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea90ea1-4c74-4bee-a6d6-c72b3f180da2 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.311517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea90ea1-4c74-4bee-a6d6-c72b3f180da2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.311997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea90ea1-4c74-4bee-a6d6-c72b3f180da2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.312868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea90ea1-4c74-4bee-a6d6-c72b3f180da2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:41.314855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea90ea1-4c74-4bee-a6d6-c72b3f180da2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:41.331646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1093/2214] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:40 2022] GET /identity/v3/roles => generated 1420 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:41.336951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:41.338708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.362909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a80a9e1-2708-413f-b776-63c1ec3b421b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:41.371153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d408b1d-b2d5-4eaf-a23f-f5376d074259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:41.372461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d408b1d-b2d5-4eaf-a23f-f5376d074259 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.373566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d408b1d-b2d5-4eaf-a23f-f5376d074259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.373566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d408b1d-b2d5-4eaf-a23f-f5376d074259 admin admin] PATH_INFO: `/v3/projects/76569a470885443abfbe4e14e9b2f505/users/baa15cc84259470f93afcfe23ef55c2f/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.410557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d408b1d-b2d5-4eaf-a23f-f5376d074259 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76569a470885443abfbe4e14e9b2f505, user_id=baa15cc84259470f93afcfe23ef55c2f, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:41.416187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d408b1d-b2d5-4eaf-a23f-f5376d074259 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:41.454326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1122/2215] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:41 2022] PUT /identity/v3/projects/76569a470885443abfbe4e14e9b2f505/users/baa15cc84259470f93afcfe23ef55c2f/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:41.464339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a5cb-ecd1-4585-82f7-45a2dd6f7ca7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.465631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a5cb-ecd1-4585-82f7-45a2dd6f7ca7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.466406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1304a5cb-ecd1-4585-82f7-45a2dd6f7ca7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.499773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d408b1d-b2d5-4eaf-a23f-f5376d074259 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:41.501756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4d408b1d-b2d5-4eaf-a23f-f5376d074259 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.519908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1304a5cb-ecd1-4585-82f7-45a2dd6f7ca7 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:41.521520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1304a5cb-ecd1-4585-82f7-45a2dd6f7ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.623725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d408b1d-b2d5-4eaf-a23f-f5376d074259 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:41.630663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:41.631381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.632332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.632332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] PATH_INFO: `/v3/projects/43736a60a6854019972fdfea348b19fb` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.633491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] RBAC: Authorizing `identity:delete_project(project_id=43736a60a6854019972fdfea348b19fb)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:41.644756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:41.679019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1094/2216] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:41.679019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304a5cb-ecd1-4585-82f7-45a2dd6f7ca7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:41.679019 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1304a5cb-ecd1-4585-82f7-45a2dd6f7ca7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.695277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1304a5cb-ecd1-4585-82f7-45a2dd6f7ca7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:41.700970 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:41.701750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.702160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.702514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] PATH_INFO: `/v3/projects/982d94fa54754ca7b12cfa4b3916eb44` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.703829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] RBAC: Authorizing `identity:delete_project(project_id=982d94fa54754ca7b12cfa4b3916eb44)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:41.706174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:41.721660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43736a60a6854019972fdfea348b19fb', 'request_id': 'req-0767fd79-22e7-466e-ade8-e79b406f438a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.728473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43736a60a6854019972fdfea348b19fb', 'request_id': 'req-0767fd79-22e7-466e-ade8-e79b406f438a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.734813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43736a60a6854019972fdfea348b19fb', 'request_id': 'req-0767fd79-22e7-466e-ade8-e79b406f438a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.747825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] The token cache is being invalidate because project 43736a60a6854019972fdfea348b19fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:41.748215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.748875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.750549 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1123/2217] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:41 2022] DELETE /identity/v3/projects/43736a60a6854019972fdfea348b19fb => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:41.754737 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:41.756078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.757137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '982d94fa54754ca7b12cfa4b3916eb44', 'request_id': 'req-2d8c546d-1c20-4d68-8275-2e9bf552d60d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.764173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '982d94fa54754ca7b12cfa4b3916eb44', 'request_id': 'req-2d8c546d-1c20-4d68-8275-2e9bf552d60d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.769881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '982d94fa54754ca7b12cfa4b3916eb44', 'request_id': 'req-2d8c546d-1c20-4d68-8275-2e9bf552d60d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.783797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] The token cache is being invalidate because project 982d94fa54754ca7b12cfa4b3916eb44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:41.784112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.784729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2d8c546d-1c20-4d68-8275-2e9bf552d60d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:41.786511 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1095/2218] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:41 2022] DELETE /identity/v3/projects/982d94fa54754ca7b12cfa4b3916eb44 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:41.787469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a012b-6778-4c9a-b39d-066298736218 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.787751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a012b-6778-4c9a-b39d-066298736218 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.788023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a012b-6778-4c9a-b39d-066298736218 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.791417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:41.797672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:41.811957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b60a012b-6778-4c9a-b39d-066298736218 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:41.813737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b60a012b-6778-4c9a-b39d-066298736218 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.821615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0767fd79-22e7-466e-ade8-e79b406f438a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:41.870705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271fe53e-e1f0-4cfc-8055-88be703dd633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:41.871324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fe53e-e1f0-4cfc-8055-88be703dd633 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.871600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fe53e-e1f0-4cfc-8055-88be703dd633 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.871869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271fe53e-e1f0-4cfc-8055-88be703dd633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.873619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-271fe53e-e1f0-4cfc-8055-88be703dd633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.876747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1096/2219] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:41.883325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491ce9a-966b-4027-b3e0-71c4c9a89293 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.883325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491ce9a-966b-4027-b3e0-71c4c9a89293 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.883325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d491ce9a-966b-4027-b3e0-71c4c9a89293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.886328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fe53e-e1f0-4cfc-8055-88be703dd633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:41.887814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271fe53e-e1f0-4cfc-8055-88be703dd633 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:41.897800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1124/2220] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:41.898905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd15aea-25d2-421b-a363-83614e010939 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.899023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d491ce9a-966b-4027-b3e0-71c4c9a89293 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:41.899184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd15aea-25d2-421b-a363-83614e010939 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.899452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd15aea-25d2-421b-a363-83614e010939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.900507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d491ce9a-966b-4027-b3e0-71c4c9a89293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.920577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1097/2221] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:41.924626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529527-4149-41ea-87b0-ab97acdfe7c4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.924947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529527-4149-41ea-87b0-ab97acdfe7c4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.925259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de529527-4149-41ea-87b0-ab97acdfe7c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.930430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9bd15aea-25d2-421b-a363-83614e010939 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:41.931992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9bd15aea-25d2-421b-a363-83614e010939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.941042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1125/2222] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:41.945183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd15aea-25d2-421b-a363-83614e010939 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:41.948206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-de529527-4149-41ea-87b0-ab97acdfe7c4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:41.949550 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-de529527-4149-41ea-87b0-ab97acdfe7c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:41.954578 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd15aea-25d2-421b-a363-83614e010939 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:41.958962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1098/2223] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:41.959695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dbca4-f85b-4c81-a625-5d5dc2e080d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:41.960304 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dbca4-f85b-4c81-a625-5d5dc2e080d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.960580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dbca4-f85b-4c81-a625-5d5dc2e080d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.960850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2dbca4-f85b-4c81-a625-5d5dc2e080d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.961501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dbca4-f85b-4c81-a625-5d5dc2e080d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:41.961808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dbca4-f85b-4c81-a625-5d5dc2e080d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:41.963015 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529527-4149-41ea-87b0-ab97acdfe7c4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:41.969162 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2dbca4-f85b-4c81-a625-5d5dc2e080d1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:41.974929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de529527-4149-41ea-87b0-ab97acdfe7c4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:41.980896 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd00a66-a213-42e4-98c2-d7d4aac42eec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:41.982134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd00a66-a213-42e4-98c2-d7d4aac42eec admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:41.982134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd00a66-a213-42e4-98c2-d7d4aac42eec admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:41.982316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd00a66-a213-42e4-98c2-d7d4aac42eec admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:41.982966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd00a66-a213-42e4-98c2-d7d4aac42eec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:41.983327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd00a66-a213-42e4-98c2-d7d4aac42eec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:41.984986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd00a66-a213-42e4-98c2-d7d4aac42eec admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:41.987070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1126/2224] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:02:41.991411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dbca4-f85b-4c81-a625-5d5dc2e080d1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.002131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1099/2225] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:02:42.004308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd00a66-a213-42e4-98c2-d7d4aac42eec admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.006772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2dbca4-f85b-4c81-a625-5d5dc2e080d1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.013870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341f580-921d-43fa-ada9-4be93f24e445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.014756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341f580-921d-43fa-ada9-4be93f24e445 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.015138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341f580-921d-43fa-ada9-4be93f24e445 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.015430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c341f580-921d-43fa-ada9-4be93f24e445 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.015531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd00a66-a213-42e4-98c2-d7d4aac42eec admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.016154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341f580-921d-43fa-ada9-4be93f24e445 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.018621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c341f580-921d-43fa-ada9-4be93f24e445 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.021001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978e06a-1d17-4f6f-bcd0-bedf4290e0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.022007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978e06a-1d17-4f6f-bcd0-bedf4290e0e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.022497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978e06a-1d17-4f6f-bcd0-bedf4290e0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.023003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b978e06a-1d17-4f6f-bcd0-bedf4290e0e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.023835 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978e06a-1d17-4f6f-bcd0-bedf4290e0e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.026498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b978e06a-1d17-4f6f-bcd0-bedf4290e0e8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.052573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1127/2226] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:41 2022] POST /identity/v3/projects => generated 387 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:42.058288 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341f580-921d-43fa-ada9-4be93f24e445 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.060695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1100/2227] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:42 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:42.064804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978e06a-1d17-4f6f-bcd0-bedf4290e0e8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.140871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c341f580-921d-43fa-ada9-4be93f24e445 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.145628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f89e68-0639-4046-b038-c0882a499174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.146297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f89e68-0639-4046-b038-c0882a499174 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.146581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f89e68-0639-4046-b038-c0882a499174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.146921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f89e68-0639-4046-b038-c0882a499174 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.147569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f89e68-0639-4046-b038-c0882a499174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.148696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b978e06a-1d17-4f6f-bcd0-bedf4290e0e8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.149612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f89e68-0639-4046-b038-c0882a499174 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.154839 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3397b6-c8f6-43a8-ad60-97dfff07b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.155541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3397b6-c8f6-43a8-ad60-97dfff07b2a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.155898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3397b6-c8f6-43a8-ad60-97dfff07b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.156245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3397b6-c8f6-43a8-ad60-97dfff07b2a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.157007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3397b6-c8f6-43a8-ad60-97dfff07b2a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.160716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3397b6-c8f6-43a8-ad60-97dfff07b2a7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.187222 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1128/2228] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:42 2022] POST /identity/v3/users => generated 294 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:42.191285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f89e68-0639-4046-b038-c0882a499174 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.197604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1101/2229] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:42 2022] POST /identity/v3/users => generated 283 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:42.201848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3397b6-c8f6-43a8-ad60-97dfff07b2a7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.204604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f89e68-0639-4046-b038-c0882a499174 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.210919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a664a-26e1-4f14-a999-08eb6b0166c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.211608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a664a-26e1-4f14-a999-08eb6b0166c8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.211885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a664a-26e1-4f14-a999-08eb6b0166c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.212264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0a664a-26e1-4f14-a999-08eb6b0166c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.212953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a664a-26e1-4f14-a999-08eb6b0166c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.214709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0a664a-26e1-4f14-a999-08eb6b0166c8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.227405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3397b6-c8f6-43a8-ad60-97dfff07b2a7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.229775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1129/2230] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:42.230999 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a664a-26e1-4f14-a999-08eb6b0166c8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.242572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0a664a-26e1-4f14-a999-08eb6b0166c8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.243106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d937d5-5c54-42ba-83e5-67baf2e9fe83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.243837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d937d5-5c54-42ba-83e5-67baf2e9fe83 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.244192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d937d5-5c54-42ba-83e5-67baf2e9fe83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.244876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d937d5-5c54-42ba-83e5-67baf2e9fe83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.247503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d937d5-5c54-42ba-83e5-67baf2e9fe83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.247996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8dd0e-7c3f-4bb7-8271-1c7b017329f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.248830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8dd0e-7c3f-4bb7-8271-1c7b017329f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.249172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8dd0e-7c3f-4bb7-8271-1c7b017329f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.249505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8dd0e-7c3f-4bb7-8271-1c7b017329f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.250293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8dd0e-7c3f-4bb7-8271-1c7b017329f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.251113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d937d5-5c54-42ba-83e5-67baf2e9fe83 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.252122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8dd0e-7c3f-4bb7-8271-1c7b017329f4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.267793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1130/2231] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:42.269029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8dd0e-7c3f-4bb7-8271-1c7b017329f4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.269372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1102/2232] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:42.272792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d937d5-5c54-42ba-83e5-67baf2e9fe83 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.283961 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8dd0e-7c3f-4bb7-8271-1c7b017329f4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.284751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d937d5-5c54-42ba-83e5-67baf2e9fe83 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.289650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774e0a0f-3161-451a-8f83-7c1577f126af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.290501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e0a0f-3161-451a-8f83-7c1577f126af admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.290627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231fb3d-c92b-47ea-9a69-b25bde24981a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.290831 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e0a0f-3161-451a-8f83-7c1577f126af admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.291206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231fb3d-c92b-47ea-9a69-b25bde24981a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.291392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774e0a0f-3161-451a-8f83-7c1577f126af admin admin] PATH_INFO: `/v3/projects/75ceabf30cca492eabf1a4f2d363f123/users/03f457524b4143c1a263582440730cd5/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.291730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231fb3d-c92b-47ea-9a69-b25bde24981a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.292016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231fb3d-c92b-47ea-9a69-b25bde24981a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.292714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231fb3d-c92b-47ea-9a69-b25bde24981a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.294410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2231fb3d-c92b-47ea-9a69-b25bde24981a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.300530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:42.307968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1103/2233] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:42.313110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231fb3d-c92b-47ea-9a69-b25bde24981a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.314686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e0a0f-3161-451a-8f83-7c1577f126af admin admin] RBAC: Authorizing `identity:create_grant(project_id=75ceabf30cca492eabf1a4f2d363f123, user_id=03f457524b4143c1a263582440730cd5, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.319049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774e0a0f-3161-451a-8f83-7c1577f126af admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.330105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231fb3d-c92b-47ea-9a69-b25bde24981a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.338071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4f2f8e-a037-4739-90ea-b8aff366e0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.340013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f2f8e-a037-4739-90ea-b8aff366e0c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.340455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f2f8e-a037-4739-90ea-b8aff366e0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.340876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4f2f8e-a037-4739-90ea-b8aff366e0c3 admin admin] PATH_INFO: `/v3/projects/57c5f928bfc8427cbfc7dd20e7a773ba/users/b7a67839b2cb41a38eef7e288dee6f12/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.344763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1131/2234] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:42 2022] PUT /identity/v3/projects/75ceabf30cca492eabf1a4f2d363f123/users/03f457524b4143c1a263582440730cd5/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:42.357409 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c73245-9979-4aef-98aa-e66970502cb8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.357790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c73245-9979-4aef-98aa-e66970502cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.358292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c73245-9979-4aef-98aa-e66970502cb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.383937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f2f8e-a037-4739-90ea-b8aff366e0c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57c5f928bfc8427cbfc7dd20e7a773ba, user_id=b7a67839b2cb41a38eef7e288dee6f12, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.388214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4f2f8e-a037-4739-90ea-b8aff366e0c3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.396597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d3c73245-9979-4aef-98aa-e66970502cb8 None None] MFA Rules not processed for user `03f457524b4143c1a263582440730cd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:42.398183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3c73245-9979-4aef-98aa-e66970502cb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:42.404169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:42.423969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1104/2235] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:42 2022] PUT /identity/v3/projects/57c5f928bfc8427cbfc7dd20e7a773ba/users/b7a67839b2cb41a38eef7e288dee6f12/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:42.437238 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af22e1-7c86-4360-b6ae-272d38c36df1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.438334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af22e1-7c86-4360-b6ae-272d38c36df1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.438709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af22e1-7c86-4360-b6ae-272d38c36df1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.520385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f6af22e1-7c86-4360-b6ae-272d38c36df1 None None] MFA Rules not processed for user `b7a67839b2cb41a38eef7e288dee6f12`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:42.522253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f6af22e1-7c86-4360-b6ae-272d38c36df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:42.570194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1132/2236] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:42 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:42.574230 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c73245-9979-4aef-98aa-e66970502cb8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.613939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1105/2237] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:42.625181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af22e1-7c86-4360-b6ae-272d38c36df1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.626904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f6af22e1-7c86-4360-b6ae-272d38c36df1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:42.683398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c73245-9979-4aef-98aa-e66970502cb8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.690917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a3c1-a4c4-4cbe-948c-5edc47d1529b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.692003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a3c1-a4c4-4cbe-948c-5edc47d1529b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.692432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a3c1-a4c4-4cbe-948c-5edc47d1529b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.693197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb4a3c1-a4c4-4cbe-948c-5edc47d1529b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.695188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a3c1-a4c4-4cbe-948c-5edc47d1529b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.697097 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb4a3c1-a4c4-4cbe-948c-5edc47d1529b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.706304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af22e1-7c86-4360-b6ae-272d38c36df1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.710327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed7672-877b-4d7f-95e0-54c467f9676b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.711037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed7672-877b-4d7f-95e0-54c467f9676b service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.711761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed7672-877b-4d7f-95e0-54c467f9676b service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.711761 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed7672-877b-4d7f-95e0-54c467f9676b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.722980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed7672-877b-4d7f-95e0-54c467f9676b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.727452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed7672-877b-4d7f-95e0-54c467f9676b service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.737195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1133/2238] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:42 2022] POST /identity/v3/projects => generated 385 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:42.741014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1106/2239] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:02:42.753494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a3c1-a4c4-4cbe-948c-5edc47d1529b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.882472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb4a3c1-a4c4-4cbe-948c-5edc47d1529b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.894927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da15bf9-eb77-40fe-922d-7fd53385a8ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.896771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da15bf9-eb77-40fe-922d-7fd53385a8ec admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.897083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da15bf9-eb77-40fe-922d-7fd53385a8ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.897346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da15bf9-eb77-40fe-922d-7fd53385a8ec admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.898001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da15bf9-eb77-40fe-922d-7fd53385a8ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.901842 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da15bf9-eb77-40fe-922d-7fd53385a8ec admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:42.913306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:42.933250 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1134/2240] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:42 2022] POST /identity/v3/users => generated 299 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:42.937976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed7672-877b-4d7f-95e0-54c467f9676b service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:42.953395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed7672-877b-4d7f-95e0-54c467f9676b service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:42.966417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf73ba-4dd1-4ba9-96d6-b27cf81c3be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:42.967208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf73ba-4dd1-4ba9-96d6-b27cf81c3be5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:42.968397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf73ba-4dd1-4ba9-96d6-b27cf81c3be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:42.971716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bf73ba-4dd1-4ba9-96d6-b27cf81c3be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:42.972475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf73ba-4dd1-4ba9-96d6-b27cf81c3be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:42.977759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bf73ba-4dd1-4ba9-96d6-b27cf81c3be5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:43.022344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1107/2241] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:42 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:43.023401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da15bf9-eb77-40fe-922d-7fd53385a8ec admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:43.036860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da15bf9-eb77-40fe-922d-7fd53385a8ec admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:43.047153 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75029-40db-4622-ac8c-1c2031826fb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:43.048093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75029-40db-4622-ac8c-1c2031826fb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:43.048407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75029-40db-4622-ac8c-1c2031826fb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:43.048709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add75029-40db-4622-ac8c-1c2031826fb4 admin admin] PATH_INFO: `/v3/projects/728702a994604151b5c2923efacf3e82/users/71f103f7828346cb9176c3c89d54391a/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:43.081393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add75029-40db-4622-ac8c-1c2031826fb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728702a994604151b5c2923efacf3e82, user_id=71f103f7828346cb9176c3c89d54391a, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:43.085182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add75029-40db-4622-ac8c-1c2031826fb4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:43.122226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1135/2242] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:43 2022] PUT /identity/v3/projects/728702a994604151b5c2923efacf3e82/users/71f103f7828346cb9176c3c89d54391a/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:43.123447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf73ba-4dd1-4ba9-96d6-b27cf81c3be5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:43.137286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bf73ba-4dd1-4ba9-96d6-b27cf81c3be5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:43.258506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1643c0c-6917-450b-bd18-e05349962c1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:43.259279 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1643c0c-6917-450b-bd18-e05349962c1b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:43.259609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1643c0c-6917-450b-bd18-e05349962c1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:43.259883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1643c0c-6917-450b-bd18-e05349962c1b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:43.260844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1643c0c-6917-450b-bd18-e05349962c1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:43.262584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1643c0c-6917-450b-bd18-e05349962c1b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:43.280084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75029-40db-4622-ac8c-1c2031826fb4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:43.280329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1108/2243] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:43 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:43.290615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add75029-40db-4622-ac8c-1c2031826fb4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:43.295583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93d1bd-8d2d-40bd-97ea-fa3d76107e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:43.296265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93d1bd-8d2d-40bd-97ea-fa3d76107e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:43.296613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93d1bd-8d2d-40bd-97ea-fa3d76107e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:43.296812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f93d1bd-8d2d-40bd-97ea-fa3d76107e57 admin admin] PATH_INFO: `/v3/projects/728702a994604151b5c2923efacf3e82/users/71f103f7828346cb9176c3c89d54391a/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:43.298770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93d1bd-8d2d-40bd-97ea-fa3d76107e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=728702a994604151b5c2923efacf3e82, user_id=71f103f7828346cb9176c3c89d54391a, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:43.301570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f93d1bd-8d2d-40bd-97ea-fa3d76107e57 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:43.321746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1136/2244] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:43 2022] PUT /identity/v3/projects/728702a994604151b5c2923efacf3e82/users/71f103f7828346cb9176c3c89d54391a/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:43.331824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f981a-5170-4ee6-b8aa-1d40138a4503 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:43.332078 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f981a-5170-4ee6-b8aa-1d40138a4503 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:43.332478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f981a-5170-4ee6-b8aa-1d40138a4503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:43.369357 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-be5f981a-5170-4ee6-b8aa-1d40138a4503 None None] MFA Rules not processed for user `71f103f7828346cb9176c3c89d54391a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:43.371405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-be5f981a-5170-4ee6-b8aa-1d40138a4503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:43.380308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:43.461147 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1109/2245] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:43 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:43.474650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93d1bd-8d2d-40bd-97ea-fa3d76107e57 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:43.476669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3f93d1bd-8d2d-40bd-97ea-fa3d76107e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:43.558855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f93d1bd-8d2d-40bd-97ea-fa3d76107e57 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:43.564497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ebdc7-ab69-4104-a873-a89e0fe7ce53 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:43.565806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ebdc7-ab69-4104-a873-a89e0fe7ce53 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:43.566189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ebdc7-ab69-4104-a873-a89e0fe7ce53 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:43.566615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ebdc7-ab69-4104-a873-a89e0fe7ce53 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:43.581693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ebdc7-ab69-4104-a873-a89e0fe7ce53 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:43.583271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ebdc7-ab69-4104-a873-a89e0fe7ce53 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:43.614762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1137/2246] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:43.717874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f981a-5170-4ee6-b8aa-1d40138a4503 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:43.817712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f981a-5170-4ee6-b8aa-1d40138a4503 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:43.826041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:43.827079 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:43.827360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:43.827821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] PATH_INFO: `/v3/users/03f457524b4143c1a263582440730cd5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:43.829274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03f457524b4143c1a263582440730cd5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:43.831722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:43.941906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03f457524b4143c1a263582440730cd5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:43.954180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03f457524b4143c1a263582440730cd5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:43.980475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03f457524b4143c1a263582440730cd5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:43.981326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03f457524b4143c1a263582440730cd5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:43.989362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03f457524b4143c1a263582440730cd5'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:43.996862 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1110/2247] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:43 2022] DELETE /identity/v3/users/03f457524b4143c1a263582440730cd5 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:44.001486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb34e7-5f3c-4f79-b553-dcf17ddbf3c9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:44.001859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb34e7-5f3c-4f79-b553-dcf17ddbf3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:44.002239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eb34e7-5f3c-4f79-b553-dcf17ddbf3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:44.034694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f8eb34e7-5f3c-4f79-b553-dcf17ddbf3c9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:44.036376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f8eb34e7-5f3c-4f79-b553-dcf17ddbf3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:44.108556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1138/2248] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:44.118574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:44.120427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:44.150219 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:44.195628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4dc7f7-f90f-499e-abaf-84a90628db68 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:44.199686 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0389f89d-ced4-49bb-9285-c2df3f2ca6f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:44.200259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389f89d-ced4-49bb-9285-c2df3f2ca6f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:44.200541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389f89d-ced4-49bb-9285-c2df3f2ca6f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:44.200815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0389f89d-ced4-49bb-9285-c2df3f2ca6f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:44.211308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389f89d-ced4-49bb-9285-c2df3f2ca6f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:44.212799 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0389f89d-ced4-49bb-9285-c2df3f2ca6f6 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:44.225150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1111/2249] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:44.364132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb34e7-5f3c-4f79-b553-dcf17ddbf3c9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:44.366557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f8eb34e7-5f3c-4f79-b553-dcf17ddbf3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:44.385814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eb34e7-5f3c-4f79-b553-dcf17ddbf3c9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:44.399407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:44.400068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:44.400348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:44.400623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] PATH_INFO: `/v3/users/f9295a3715d947e1a08b8284829e5918` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:44.402146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9295a3715d947e1a08b8284829e5918)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:44.404495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:44.488100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9295a3715d947e1a08b8284829e5918'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:44.490003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9295a3715d947e1a08b8284829e5918'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:44.504397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=5,keystone:SELECT=87,keystone:DELETE=8 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:44.516251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9295a3715d947e1a08b8284829e5918'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:44.535767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9295a3715d947e1a08b8284829e5918'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:44.537528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9295a3715d947e1a08b8284829e5918'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:44.562811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1139/2250] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:44 2022] DELETE /identity/v3/users/f9295a3715d947e1a08b8284829e5918 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:44.567587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bde53-0792-4629-a675-f7ce0ec1af13 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:44.567913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bde53-0792-4629-a675-f7ce0ec1af13 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:44.568432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29bde53-0792-4629-a675-f7ce0ec1af13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:44.601653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f29bde53-0792-4629-a675-f7ce0ec1af13 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:44.603631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f29bde53-0792-4629-a675-f7ce0ec1af13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:44.704373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1112/2251] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:44.719470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:44.721130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:44.811214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36756809-a01e-4cb5-ba54-f9c1b6d4aa5b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:44.816001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0138181-0d04-42ac-bd55-88440f6b6e08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:44.817025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0138181-0d04-42ac-bd55-88440f6b6e08 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:44.817685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0138181-0d04-42ac-bd55-88440f6b6e08 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:44.818956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0138181-0d04-42ac-bd55-88440f6b6e08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:44.835857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bde53-0792-4629-a675-f7ce0ec1af13 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:44.836387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0138181-0d04-42ac-bd55-88440f6b6e08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:44.837973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f29bde53-0792-4629-a675-f7ce0ec1af13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:44.839509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0138181-0d04-42ac-bd55-88440f6b6e08 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:44.867264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1140/2252] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:44.882747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29bde53-0792-4629-a675-f7ce0ec1af13 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:44.898602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:44.899282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:44.899556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:44.899822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] PATH_INFO: `/v3/projects/75ceabf30cca492eabf1a4f2d363f123` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:44.902029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] RBAC: Authorizing `identity:delete_project(project_id=75ceabf30cca492eabf1a4f2d363f123)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:44.903995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:45.003651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75ceabf30cca492eabf1a4f2d363f123', 'request_id': 'req-b56c238e-9063-42d6-8616-4d024073338d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.014484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75ceabf30cca492eabf1a4f2d363f123', 'request_id': 'req-b56c238e-9063-42d6-8616-4d024073338d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.026928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75ceabf30cca492eabf1a4f2d363f123', 'request_id': 'req-b56c238e-9063-42d6-8616-4d024073338d'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.087103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] The token cache is being invalidate because project 75ceabf30cca492eabf1a4f2d363f123 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:45.087722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.095439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.097895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1113/2253] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:44 2022] DELETE /identity/v3/projects/75ceabf30cca492eabf1a4f2d363f123 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:45.102438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0138181-0d04-42ac-bd55-88440f6b6e08 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:45.117649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0138181-0d04-42ac-bd55-88440f6b6e08 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:45.286208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:45.287358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:45.287964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:45.288339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] PATH_INFO: `/v3/users/71f103f7828346cb9176c3c89d54391a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:45.290861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] RBAC: Authorizing `identity:delete_user(user_id=71f103f7828346cb9176c3c89d54391a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:45.294431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:45.411227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '71f103f7828346cb9176c3c89d54391a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.412472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '71f103f7828346cb9176c3c89d54391a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.424577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '71f103f7828346cb9176c3c89d54391a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.443213 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '71f103f7828346cb9176c3c89d54391a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.444427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '71f103f7828346cb9176c3c89d54391a'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.449953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:45.472881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1141/2254] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:45 2022] DELETE /identity/v3/users/71f103f7828346cb9176c3c89d54391a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:45.585368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:45.587204 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:45.606891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56c238e-9063-42d6-8616-4d024073338d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:45.669509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:45.710071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:45.710863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:45.711170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:45.713667 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] PATH_INFO: `/v3/projects/c5c726d5677c46e5b4fdbdf246d74c46` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:45.716127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5c726d5677c46e5b4fdbdf246d74c46)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:45.721121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:45.822377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5c726d5677c46e5b4fdbdf246d74c46', 'request_id': 'req-00b2525c-74da-4341-b58d-185cccb68129'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.830748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5c726d5677c46e5b4fdbdf246d74c46', 'request_id': 'req-00b2525c-74da-4341-b58d-185cccb68129'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.843426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5c726d5677c46e5b4fdbdf246d74c46', 'request_id': 'req-00b2525c-74da-4341-b58d-185cccb68129'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.868087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] The token cache is being invalidate because project c5c726d5677c46e5b4fdbdf246d74c46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:45.868439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.870115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:45.872174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1114/2255] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:45 2022] DELETE /identity/v3/projects/c5c726d5677c46e5b4fdbdf246d74c46 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:45.875670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:45.878429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:45.903861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50da6fc4-d55f-469e-81b1-222feca2f36b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:45.924951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:45.926412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:46.023263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b2525c-74da-4341-b58d-185cccb68129 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:46.028804 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:46.029466 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:46.029754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:46.030032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] PATH_INFO: `/v3/projects/728702a994604151b5c2923efacf3e82` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:46.031222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] RBAC: Authorizing `identity:delete_project(project_id=728702a994604151b5c2923efacf3e82)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:46.034135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:46.074754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:46.076754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:46.078656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:46.079045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] PATH_INFO: `/v3/users/5fb05841214a46f8954f12472b866bb7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:46.082471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fb05841214a46f8954f12472b866bb7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:46.087401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:46.123472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '728702a994604151b5c2923efacf3e82', 'request_id': 'req-caa693c4-6e96-4597-97b1-072ca8e88641'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.133375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '728702a994604151b5c2923efacf3e82', 'request_id': 'req-caa693c4-6e96-4597-97b1-072ca8e88641'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.143195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '728702a994604151b5c2923efacf3e82', 'request_id': 'req-caa693c4-6e96-4597-97b1-072ca8e88641'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.163030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] The token cache is being invalidate because project 728702a994604151b5c2923efacf3e82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:46.163344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.164008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-caa693c4-6e96-4597-97b1-072ca8e88641 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.169206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1115/2256] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:45 2022] DELETE /identity/v3/projects/728702a994604151b5c2923efacf3e82 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:46.174606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d020694-37d6-4407-a3f7-2e16a4fd079c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:46.174897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d020694-37d6-4407-a3f7-2e16a4fd079c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:46.175178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d020694-37d6-4407-a3f7-2e16a4fd079c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:46.176938 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fb05841214a46f8954f12472b866bb7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.177865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fb05841214a46f8954f12472b866bb7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.195060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fb05841214a46f8954f12472b866bb7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.223448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fb05841214a46f8954f12472b866bb7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.224722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e47d7f26-3dad-47d6-ae60-acf8f46d2cb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fb05841214a46f8954f12472b866bb7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:46.230931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1d020694-37d6-4407-a3f7-2e16a4fd079c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:46.233009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1d020694-37d6-4407-a3f7-2e16a4fd079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:46.251778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:46.257447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1142/2257] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:45 2022] DELETE /identity/v3/users/5fb05841214a46f8954f12472b866bb7 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:46.424235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1116/2258] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:46.426251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4000-da9e-4c8c-b728-e5f6d244f461 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:46.426608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4000-da9e-4c8c-b728-e5f6d244f461 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:46.427000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429a4000-da9e-4c8c-b728-e5f6d244f461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:46.468315 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-429a4000-da9e-4c8c-b728-e5f6d244f461 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:46.472474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-429a4000-da9e-4c8c-b728-e5f6d244f461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:46.578423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1143/2259] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:46.580520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d020694-37d6-4407-a3f7-2e16a4fd079c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:46.625944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d020694-37d6-4407-a3f7-2e16a4fd079c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:46.640545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790e5eb-0705-4787-b72f-0657fef32230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:46.641210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790e5eb-0705-4787-b72f-0657fef32230 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:46.641477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790e5eb-0705-4787-b72f-0657fef32230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:46.641754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790e5eb-0705-4787-b72f-0657fef32230 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:46.643309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790e5eb-0705-4787-b72f-0657fef32230 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:46.643675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790e5eb-0705-4787-b72f-0657fef32230 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:46.645595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790e5eb-0705-4787-b72f-0657fef32230 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:46.695584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1117/2260] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:46.700564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a4000-da9e-4c8c-b728-e5f6d244f461 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:46.715623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429a4000-da9e-4c8c-b728-e5f6d244f461 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:46.727309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338147e-5dbe-43f9-95f7-56cc1b35a64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:46.727950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338147e-5dbe-43f9-95f7-56cc1b35a64b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:46.728227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338147e-5dbe-43f9-95f7-56cc1b35a64b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:46.728500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4338147e-5dbe-43f9-95f7-56cc1b35a64b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:46.729227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338147e-5dbe-43f9-95f7-56cc1b35a64b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:46.733470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4338147e-5dbe-43f9-95f7-56cc1b35a64b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:46.789647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:46.845053 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1144/2261] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:46 2022] POST /identity/v3/projects => generated 381 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:46.849233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790e5eb-0705-4787-b72f-0657fef32230 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:46.869620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790e5eb-0705-4787-b72f-0657fef32230 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.019068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b86715-a19d-4769-a747-a99132b13d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.020134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b86715-a19d-4769-a747-a99132b13d9c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.020637 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b86715-a19d-4769-a747-a99132b13d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.021130 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b86715-a19d-4769-a747-a99132b13d9c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.022168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b86715-a19d-4769-a747-a99132b13d9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:47.025176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b86715-a19d-4769-a747-a99132b13d9c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:47.076455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338147e-5dbe-43f9-95f7-56cc1b35a64b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:47.082124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1118/2262] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:46 2022] POST /identity/v3/users => generated 291 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:47.087360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4338147e-5dbe-43f9-95f7-56cc1b35a64b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.116371 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f0659-e3d2-4eb2-8c0f-e0e60ea2f98b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.117031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f0659-e3d2-4eb2-8c0f-e0e60ea2f98b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.117318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f0659-e3d2-4eb2-8c0f-e0e60ea2f98b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.120098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06f0659-e3d2-4eb2-8c0f-e0e60ea2f98b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.120636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f0659-e3d2-4eb2-8c0f-e0e60ea2f98b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:47.122345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06f0659-e3d2-4eb2-8c0f-e0e60ea2f98b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:47.143693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1145/2263] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:47.146069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b86715-a19d-4769-a747-a99132b13d9c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:47.168026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b86715-a19d-4769-a747-a99132b13d9c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.175287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5835205-1430-480b-a73e-1cd30efa16ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.176119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5835205-1430-480b-a73e-1cd30efa16ff admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.176477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5835205-1430-480b-a73e-1cd30efa16ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.176787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5835205-1430-480b-a73e-1cd30efa16ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.178326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5835205-1430-480b-a73e-1cd30efa16ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:47.184535 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5835205-1430-480b-a73e-1cd30efa16ff admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:47.190723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f0659-e3d2-4eb2-8c0f-e0e60ea2f98b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:47.192596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b06f0659-e3d2-4eb2-8c0f-e0e60ea2f98b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:47.204069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1119/2264] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:47.205077 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5835205-1430-480b-a73e-1cd30efa16ff admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:47.216093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06f0659-e3d2-4eb2-8c0f-e0e60ea2f98b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.223353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.224902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.225893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5835205-1430-480b-a73e-1cd30efa16ff admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.226654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.226975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] PATH_INFO: `/v3/projects/1af8464269bc492e9ad1c2ff812e0eea` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.228373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1af8464269bc492e9ad1c2ff812e0eea)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:47.232060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:47.233773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e00535-5d63-4909-9504-6ef8c986b938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.235376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e00535-5d63-4909-9504-6ef8c986b938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.236105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e00535-5d63-4909-9504-6ef8c986b938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.236616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e00535-5d63-4909-9504-6ef8c986b938 admin admin] PATH_INFO: `/v3/projects/6dfbc1dfc78f437b8722404536e8c5fd/users/49294a2924e944a7b5fcca32c2e8683b/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.257174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e00535-5d63-4909-9504-6ef8c986b938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dfbc1dfc78f437b8722404536e8c5fd, user_id=49294a2924e944a7b5fcca32c2e8683b, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:47.260656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e00535-5d63-4909-9504-6ef8c986b938 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:47.286864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1120/2265] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:47 2022] PUT /identity/v3/projects/6dfbc1dfc78f437b8722404536e8c5fd/users/49294a2924e944a7b5fcca32c2e8683b/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:47.294649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f187a6c-23a8-49b8-b1f3-23c180b6e86d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.295983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f187a6c-23a8-49b8-b1f3-23c180b6e86d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.296420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f187a6c-23a8-49b8-b1f3-23c180b6e86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.299303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1af8464269bc492e9ad1c2ff812e0eea', 'request_id': 'req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.311897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1af8464269bc492e9ad1c2ff812e0eea', 'request_id': 'req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.323342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1af8464269bc492e9ad1c2ff812e0eea', 'request_id': 'req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.340358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] The token cache is being invalidate because project 1af8464269bc492e9ad1c2ff812e0eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:47.340810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.341828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.342950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3f187a6c-23a8-49b8-b1f3-23c180b6e86d None None] MFA Rules not processed for user `49294a2924e944a7b5fcca32c2e8683b`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:47.344441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3f187a6c-23a8-49b8-b1f3-23c180b6e86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:47.344677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1146/2266] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:47 2022] DELETE /identity/v3/projects/1af8464269bc492e9ad1c2ff812e0eea => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:47.348464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:47.350469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:47.405729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:47.453986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbb3d0-7ff6-425a-b39f-23363d0f8ad3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.460190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.460996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.461306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.461650 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] PATH_INFO: `/v3/users/5b28b21dede045f980246eb11aa47937` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.462862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5b28b21dede045f980246eb11aa47937)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:47.464345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1121/2267] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:47.468995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:47.470376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f187a6c-23a8-49b8-b1f3-23c180b6e86d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:47.473326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3f187a6c-23a8-49b8-b1f3-23c180b6e86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:47.504467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f187a6c-23a8-49b8-b1f3-23c180b6e86d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.516512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3abed3-e1de-464a-b264-e411377e8e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.517320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3abed3-e1de-464a-b264-e411377e8e93 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.517666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3abed3-e1de-464a-b264-e411377e8e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.518163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3abed3-e1de-464a-b264-e411377e8e93 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.518910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3abed3-e1de-464a-b264-e411377e8e93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:47.521107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3abed3-e1de-464a-b264-e411377e8e93 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:47.558841 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5b28b21dede045f980246eb11aa47937'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.559909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5b28b21dede045f980246eb11aa47937'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.562156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1122/2268] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:47 2022] POST /identity/v3/projects => generated 381 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:47.565682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3abed3-e1de-464a-b264-e411377e8e93 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:47.574320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5b28b21dede045f980246eb11aa47937'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.598732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=81,keystone:INSERT=3,keystone:DELETE=16 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:47.604129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5b28b21dede045f980246eb11aa47937'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.605227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5b28b21dede045f980246eb11aa47937'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:47.645696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1147/2269] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:47 2022] DELETE /identity/v3/users/5b28b21dede045f980246eb11aa47937 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:47.715697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3abed3-e1de-464a-b264-e411377e8e93 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.817241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954772d6-5c36-44a3-b861-ece146082a8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.817241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954772d6-5c36-44a3-b861-ece146082a8f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.817241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954772d6-5c36-44a3-b861-ece146082a8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.817241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954772d6-5c36-44a3-b861-ece146082a8f admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.819077 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954772d6-5c36-44a3-b861-ece146082a8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:47.825344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954772d6-5c36-44a3-b861-ece146082a8f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:47.860299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1123/2270] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:47 2022] POST /identity/v3/users => generated 297 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:47.862458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:47.864461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:47.884704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e71023-51c6-4aa0-8015-14d12392c8d5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.890969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55f15-8ec9-4811-b914-ecfd07648011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.891995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55f15-8ec9-4811-b914-ecfd07648011 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.892487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55f15-8ec9-4811-b914-ecfd07648011 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.892965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d55f15-8ec9-4811-b914-ecfd07648011 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:47.893826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55f15-8ec9-4811-b914-ecfd07648011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:47.896420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d55f15-8ec9-4811-b914-ecfd07648011 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:47.914712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1148/2271] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:47.915944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954772d6-5c36-44a3-b861-ece146082a8f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:47.938354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954772d6-5c36-44a3-b861-ece146082a8f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:47.943926 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dadd7-dd2e-42b2-91ee-7e77f2beec96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:47.944705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dadd7-dd2e-42b2-91ee-7e77f2beec96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:47.945271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dadd7-dd2e-42b2-91ee-7e77f2beec96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:47.945956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51dadd7-dd2e-42b2-91ee-7e77f2beec96 admin admin] PATH_INFO: `/v3/projects/1e85c01c0f054a3a90c46870f96cffe3/users/a3335b5fac2e4a9aa1260a88b0a94543/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.008217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51dadd7-dd2e-42b2-91ee-7e77f2beec96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e85c01c0f054a3a90c46870f96cffe3, user_id=a3335b5fac2e4a9aa1260a88b0a94543, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.012411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51dadd7-dd2e-42b2-91ee-7e77f2beec96 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.030675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1124/2272] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:47 2022] PUT /identity/v3/projects/1e85c01c0f054a3a90c46870f96cffe3/users/a3335b5fac2e4a9aa1260a88b0a94543/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:48.035198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55f15-8ec9-4811-b914-ecfd07648011 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.076646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d55f15-8ec9-4811-b914-ecfd07648011 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.184271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dadd7-dd2e-42b2-91ee-7e77f2beec96 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.186054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c51dadd7-dd2e-42b2-91ee-7e77f2beec96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:48.197320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda92835-d227-4b00-9dc6-36c75a3c673f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.197320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda92835-d227-4b00-9dc6-36c75a3c673f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.197320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda92835-d227-4b00-9dc6-36c75a3c673f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.197320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda92835-d227-4b00-9dc6-36c75a3c673f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.199323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda92835-d227-4b00-9dc6-36c75a3c673f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.201718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda92835-d227-4b00-9dc6-36c75a3c673f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.206303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51dadd7-dd2e-42b2-91ee-7e77f2beec96 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.224991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1149/2273] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:48.226497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9efca-c16c-4e8a-a75f-8e7dba7511ad None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.226770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9efca-c16c-4e8a-a75f-8e7dba7511ad None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.227050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f9efca-c16c-4e8a-a75f-8e7dba7511ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.227338 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.228305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.228762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.230534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] PATH_INFO: `/v3/projects/0ccc330bca994055a085fb0c9e6679ac` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.232269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ccc330bca994055a085fb0c9e6679ac)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.234795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.266508 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-11f9efca-c16c-4e8a-a75f-8e7dba7511ad None None] MFA Rules not processed for user `baa15cc84259470f93afcfe23ef55c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:48.267738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-11f9efca-c16c-4e8a-a75f-8e7dba7511ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:48.293507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:48.324510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ccc330bca994055a085fb0c9e6679ac', 'request_id': 'req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:48.332312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ccc330bca994055a085fb0c9e6679ac', 'request_id': 'req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:48.342477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ccc330bca994055a085fb0c9e6679ac', 'request_id': 'req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:48.363026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] The token cache is being invalidate because project 0ccc330bca994055a085fb0c9e6679ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:48.363434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:48.364182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:48.366774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1125/2274] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:48 2022] DELETE /identity/v3/projects/0ccc330bca994055a085fb0c9e6679ac => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:48.373208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1150/2275] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:48 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:48.373837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.373971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f87307-b29d-4790-83d6-a771d2507817 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.373971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f87307-b29d-4790-83d6-a771d2507817 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.373971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f87307-b29d-4790-83d6-a771d2507817 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.375901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:48.396029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-58f87307-b29d-4790-83d6-a771d2507817 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:48.397347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-58f87307-b29d-4790-83d6-a771d2507817 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:48.450907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97537915-d2c4-4e2f-b4d9-04b9afc7efdb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.457507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf49f5f-30fe-4195-a8d4-856508197b02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.458407 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf49f5f-30fe-4195-a8d4-856508197b02 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.458696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf49f5f-30fe-4195-a8d4-856508197b02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.458960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf49f5f-30fe-4195-a8d4-856508197b02 admin admin] PATH_INFO: `/v3/projects/1e85c01c0f054a3a90c46870f96cffe3/users/a3335b5fac2e4a9aa1260a88b0a94543/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.461671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf49f5f-30fe-4195-a8d4-856508197b02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e85c01c0f054a3a90c46870f96cffe3, user_id=a3335b5fac2e4a9aa1260a88b0a94543, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.463830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:48.465093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf49f5f-30fe-4195-a8d4-856508197b02 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.478994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1126/2276] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:48 2022] PUT /identity/v3/projects/1e85c01c0f054a3a90c46870f96cffe3/users/a3335b5fac2e4a9aa1260a88b0a94543/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:48.480631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf49f5f-30fe-4195-a8d4-856508197b02 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.480745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1151/2277] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:48.491178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa013b-39c4-4286-a772-caf9cd87ca83 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.491178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa013b-39c4-4286-a772-caf9cd87ca83 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.491178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afa013b-39c4-4286-a772-caf9cd87ca83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.491428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1bf49f5f-30fe-4195-a8d4-856508197b02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:48.521974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6afa013b-39c4-4286-a772-caf9cd87ca83 None None] MFA Rules not processed for user `a3335b5fac2e4a9aa1260a88b0a94543`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:48.523214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6afa013b-39c4-4286-a772-caf9cd87ca83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:48.581413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf49f5f-30fe-4195-a8d4-856508197b02 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.587878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262e3cf-6d89-4f39-a5a7-8b751dda8881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.589120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262e3cf-6d89-4f39-a5a7-8b751dda8881 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.589495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262e3cf-6d89-4f39-a5a7-8b751dda8881 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.589660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c262e3cf-6d89-4f39-a5a7-8b751dda8881 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.590364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262e3cf-6d89-4f39-a5a7-8b751dda8881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.596928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c262e3cf-6d89-4f39-a5a7-8b751dda8881 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.620050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1152/2278] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:48 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:48.621496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add717e-17f3-4765-b43a-641b58098370 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.621789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add717e-17f3-4765-b43a-641b58098370 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.622129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8add717e-17f3-4765-b43a-641b58098370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.647393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1127/2279] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:48 2022] POST /identity/v3/projects => generated 383 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:48.648541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262e3cf-6d89-4f39-a5a7-8b751dda8881 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.650571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c262e3cf-6d89-4f39-a5a7-8b751dda8881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:48.652244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8add717e-17f3-4765-b43a-641b58098370 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:48.653460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8add717e-17f3-4765-b43a-641b58098370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:48.718135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:48.738259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1153/2280] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:48.741913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add717e-17f3-4765-b43a-641b58098370 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.756418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8add717e-17f3-4765-b43a-641b58098370 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.762279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeac632-b1a8-4de6-82e9-d369f9ea0526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.762920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeac632-b1a8-4de6-82e9-d369f9ea0526 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.763201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeac632-b1a8-4de6-82e9-d369f9ea0526 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.763502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abeac632-b1a8-4de6-82e9-d369f9ea0526 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.764238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeac632-b1a8-4de6-82e9-d369f9ea0526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.766244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abeac632-b1a8-4de6-82e9-d369f9ea0526 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.775853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c262e3cf-6d89-4f39-a5a7-8b751dda8881 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.781074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17b46c-7316-4f97-962b-0b979208f7f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.781840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17b46c-7316-4f97-962b-0b979208f7f2 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.782269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17b46c-7316-4f97-962b-0b979208f7f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.782670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e17b46c-7316-4f97-962b-0b979208f7f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.804076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1154/2281] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:48 2022] POST /identity/v3/users => generated 298 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:48.805180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeac632-b1a8-4de6-82e9-d369f9ea0526 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.816653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abeac632-b1a8-4de6-82e9-d369f9ea0526 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.830579 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f307f65-5a9e-48a9-acc8-c639ff15ff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.831171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307f65-5a9e-48a9-acc8-c639ff15ff6e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.831538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307f65-5a9e-48a9-acc8-c639ff15ff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.831851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f307f65-5a9e-48a9-acc8-c639ff15ff6e admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.832944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f307f65-5a9e-48a9-acc8-c639ff15ff6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:48.833408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f307f65-5a9e-48a9-acc8-c639ff15ff6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.835285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f307f65-5a9e-48a9-acc8-c639ff15ff6e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.857322 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1155/2282] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:48.858184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f307f65-5a9e-48a9-acc8-c639ff15ff6e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.869244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f307f65-5a9e-48a9-acc8-c639ff15ff6e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.874633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db6805-04f1-494b-9acd-852fbf2535f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.875472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db6805-04f1-494b-9acd-852fbf2535f5 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.875902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db6805-04f1-494b-9acd-852fbf2535f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.876277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95db6805-04f1-494b-9acd-852fbf2535f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.877039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db6805-04f1-494b-9acd-852fbf2535f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.879234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95db6805-04f1-494b-9acd-852fbf2535f5 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.891454 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17b46c-7316-4f97-962b-0b979208f7f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.892904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e17b46c-7316-4f97-962b-0b979208f7f2 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.895864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1156/2283] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:48.900123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db6805-04f1-494b-9acd-852fbf2535f5 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.905190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1128/2284] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:48 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:48.906134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17b46c-7316-4f97-962b-0b979208f7f2 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.912286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95db6805-04f1-494b-9acd-852fbf2535f5 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.918182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f6b87-0f48-4141-b101-33df2e467f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.918957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f6b87-0f48-4141-b101-33df2e467f09 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.919331 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f6b87-0f48-4141-b101-33df2e467f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.919704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f6b87-0f48-4141-b101-33df2e467f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.920413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e17b46c-7316-4f97-962b-0b979208f7f2 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:48.920544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f6b87-0f48-4141-b101-33df2e467f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.922601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f6b87-0f48-4141-b101-33df2e467f09 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.926741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50fba9-5e3d-4032-ab69-02f19f0c4666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:48.927424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50fba9-5e3d-4032-ab69-02f19f0c4666 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:48.927706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50fba9-5e3d-4032-ab69-02f19f0c4666 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:48.927978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c50fba9-5e3d-4032-ab69-02f19f0c4666 admin admin] PATH_INFO: `/v3/projects/f866d57a23e44b9ca8a9349a95bc6020/users/2e34c55dc58a44ada90803e5530ab447/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:48.957609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50fba9-5e3d-4032-ab69-02f19f0c4666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f866d57a23e44b9ca8a9349a95bc6020, user_id=2e34c55dc58a44ada90803e5530ab447, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:48.961088 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c50fba9-5e3d-4032-ab69-02f19f0c4666 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:48.975910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1157/2285] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:48 2022] POST /identity/v3/projects => generated 385 bytes in 76 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:48.985086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f6b87-0f48-4141-b101-33df2e467f09 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:48.985440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1129/2286] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:48 2022] PUT /identity/v3/projects/f866d57a23e44b9ca8a9349a95bc6020/users/2e34c55dc58a44ada90803e5530ab447/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:48.990232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50fba9-5e3d-4032-ab69-02f19f0c4666 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:49.024158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:49.110844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f6b87-0f48-4141-b101-33df2e467f09 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:49.114521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f6035-35ee-4c2e-9ff9-b506ef3d32b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:49.115206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f6035-35ee-4c2e-9ff9-b506ef3d32b2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:49.115492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f6035-35ee-4c2e-9ff9-b506ef3d32b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:49.115771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9f6035-35ee-4c2e-9ff9-b506ef3d32b2 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:49.116456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f6035-35ee-4c2e-9ff9-b506ef3d32b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:49.119622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9f6035-35ee-4c2e-9ff9-b506ef3d32b2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:49.132754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c50fba9-5e3d-4032-ab69-02f19f0c4666 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:49.141652 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82885a0f-6bf8-44f8-b26c-b1a5ac386553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:49.142502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82885a0f-6bf8-44f8-b26c-b1a5ac386553 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:49.142896 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82885a0f-6bf8-44f8-b26c-b1a5ac386553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:49.143227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82885a0f-6bf8-44f8-b26c-b1a5ac386553 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:49.144341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82885a0f-6bf8-44f8-b26c-b1a5ac386553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:49.149909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82885a0f-6bf8-44f8-b26c-b1a5ac386553 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:49.159601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1158/2287] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:48 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:49.163383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f6035-35ee-4c2e-9ff9-b506ef3d32b2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:49.167414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1130/2288] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:48 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:49.168458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82885a0f-6bf8-44f8-b26c-b1a5ac386553 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:49.181983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82885a0f-6bf8-44f8-b26c-b1a5ac386553 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:49.186981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9f6035-35ee-4c2e-9ff9-b506ef3d32b2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:49.188345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6ea3c-e452-4881-b1fd-5509912d94e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:49.189036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6ea3c-e452-4881-b1fd-5509912d94e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:49.189314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6ea3c-e452-4881-b1fd-5509912d94e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:49.189692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6ea3c-e452-4881-b1fd-5509912d94e8 admin admin] PATH_INFO: `/v3/projects/f866d57a23e44b9ca8a9349a95bc6020/users/2e34c55dc58a44ada90803e5530ab447/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:49.194945 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6ea3c-e452-4881-b1fd-5509912d94e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f866d57a23e44b9ca8a9349a95bc6020, user_id=2e34c55dc58a44ada90803e5530ab447, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:49.197504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948170cd-6115-481c-b4ef-31e55c04bc4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:49.197992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6ea3c-e452-4881-b1fd-5509912d94e8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:49.199693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948170cd-6115-481c-b4ef-31e55c04bc4e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:49.199977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948170cd-6115-481c-b4ef-31e55c04bc4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:49.204416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948170cd-6115-481c-b4ef-31e55c04bc4e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:49.205188 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948170cd-6115-481c-b4ef-31e55c04bc4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:49.207078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948170cd-6115-481c-b4ef-31e55c04bc4e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:49.226126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1159/2289] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:49.228359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948170cd-6115-481c-b4ef-31e55c04bc4e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:49.240097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1131/2290] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:49 2022] PUT /identity/v3/projects/f866d57a23e44b9ca8a9349a95bc6020/users/2e34c55dc58a44ada90803e5530ab447/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:49.241562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948170cd-6115-481c-b4ef-31e55c04bc4e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:49.349770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf0fe2-4783-418a-bffa-2bda8a7ae8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:49.352452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf0fe2-4783-418a-bffa-2bda8a7ae8be admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:49.352853 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf0fe2-4783-418a-bffa-2bda8a7ae8be admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:49.353233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf0fe2-4783-418a-bffa-2bda8a7ae8be admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:49.354058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf0fe2-4783-418a-bffa-2bda8a7ae8be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:49.359147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aaf0fe2-4783-418a-bffa-2bda8a7ae8be admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:49.404147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1160/2291] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:49 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:49.405596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6ea3c-e452-4881-b1fd-5509912d94e8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:49.442009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6ea3c-e452-4881-b1fd-5509912d94e8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:49.474801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404d73fb-d119-4056-8fe8-0164e3841c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:49.475517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d73fb-d119-4056-8fe8-0164e3841c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:49.480539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d73fb-d119-4056-8fe8-0164e3841c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:49.480887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404d73fb-d119-4056-8fe8-0164e3841c4e admin admin] PATH_INFO: `/v3/projects/2da60689edf04a8f946ca07f9ae268b8/users/1389f526e0d5490f9ee485415b6b49d6/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:49.553349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf0fe2-4783-418a-bffa-2bda8a7ae8be admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:49.568477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d73fb-d119-4056-8fe8-0164e3841c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=2da60689edf04a8f946ca07f9ae268b8, user_id=1389f526e0d5490f9ee485415b6b49d6, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:49.583598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404d73fb-d119-4056-8fe8-0164e3841c4e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:49.623522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1132/2292] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:49 2022] PUT /identity/v3/projects/2da60689edf04a8f946ca07f9ae268b8/users/1389f526e0d5490f9ee485415b6b49d6/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:02:49.701082 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf0fe2-4783-418a-bffa-2bda8a7ae8be admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:49.714277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8ce81-a675-4347-98a9-203f0ae8b025 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:49.714708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8ce81-a675-4347-98a9-203f0ae8b025 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:49.715112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd8ce81-a675-4347-98a9-203f0ae8b025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:49.750321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:49.777498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3bd8ce81-a675-4347-98a9-203f0ae8b025 None None] MFA Rules not processed for user `1389f526e0d5490f9ee485415b6b49d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:49.779574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3bd8ce81-a675-4347-98a9-203f0ae8b025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:49.787431 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:49.787806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec707a1-7c39-4698-a67d-e80e3ac77be7 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3335b5fac2e4a9aa1260a88b0a94543', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e85c01c0f054a3a90c46870f96cffe3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:49.788436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec707a1-7c39-4698-a67d-e80e3ac77be7 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:49.788702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec707a1-7c39-4698-a67d-e80e3ac77be7 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:49.788975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec707a1-7c39-4698-a67d-e80e3ac77be7 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:49.789668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec707a1-7c39-4698-a67d-e80e3ac77be7 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:49.791124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec707a1-7c39-4698-a67d-e80e3ac77be7 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:49.793473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec707a1-7c39-4698-a67d-e80e3ac77be7 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:49.813360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1161/2293] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:49.822755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec707a1-7c39-4698-a67d-e80e3ac77be7 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:49.841381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec707a1-7c39-4698-a67d-e80e3ac77be7 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:49.846742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434a4ec-02df-4291-8d8d-5aae7a94c838 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3335b5fac2e4a9aa1260a88b0a94543', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e85c01c0f054a3a90c46870f96cffe3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:49.847498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434a4ec-02df-4291-8d8d-5aae7a94c838 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:49.847784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434a4ec-02df-4291-8d8d-5aae7a94c838 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:49.848151 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434a4ec-02df-4291-8d8d-5aae7a94c838 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:49.849066 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434a4ec-02df-4291-8d8d-5aae7a94c838 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:49.861109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7434a4ec-02df-4291-8d8d-5aae7a94c838 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:49.900426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1133/2294] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:49 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:02:49.916156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1162/2295] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:49 2022] POST /identity/v3/projects => generated 354 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:49.917343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8ce81-a675-4347-98a9-203f0ae8b025 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:49.919646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3bd8ce81-a675-4347-98a9-203f0ae8b025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:49.921465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434a4ec-02df-4291-8d8d-5aae7a94c838 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:50.009368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd8ce81-a675-4347-98a9-203f0ae8b025 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:50.024185 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7434a4ec-02df-4291-8d8d-5aae7a94c838 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:50.030058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250042e-6884-42b6-9480-889ca0d8a439 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3335b5fac2e4a9aa1260a88b0a94543', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e85c01c0f054a3a90c46870f96cffe3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:50.030741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250042e-6884-42b6-9480-889ca0d8a439 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:50.031031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250042e-6884-42b6-9480-889ca0d8a439 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:50.031310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8250042e-6884-42b6-9480-889ca0d8a439 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:50.031972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250042e-6884-42b6-9480-889ca0d8a439 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:50.032285 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250042e-6884-42b6-9480-889ca0d8a439 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:50.034161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8250042e-6884-42b6-9480-889ca0d8a439 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:50.061139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1163/2296] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:50.091221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824c9cc-4bb8-4b27-bc26-0377bba9f841 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:50.092120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824c9cc-4bb8-4b27-bc26-0377bba9f841 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:50.092481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824c9cc-4bb8-4b27-bc26-0377bba9f841 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:50.093181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6824c9cc-4bb8-4b27-bc26-0377bba9f841 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:50.122975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250042e-6884-42b6-9480-889ca0d8a439 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:50.134896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8250042e-6884-42b6-9480-889ca0d8a439 tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:50.141202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3335b5fac2e4a9aa1260a88b0a94543', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1e85c01c0f054a3a90c46870f96cffe3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:50.141934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:50.142364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:50.142911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] PATH_INFO: `/v3/projects/42448e35ec854b4e8e62658f5722ec38` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:50.144549 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Authorizing `identity:delete_project(project_id=42448e35ec854b4e8e62658f5722ec38)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:50.146992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:50.203742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824c9cc-4bb8-4b27-bc26-0377bba9f841 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:50.205262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6824c9cc-4bb8-4b27-bc26-0377bba9f841 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:50.215197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42448e35ec854b4e8e62658f5722ec38', 'request_id': 'req-89be1bee-d0e4-420e-963e-ee9bfd70e05c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:50.216276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1134/2297] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:49 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:50.222326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42448e35ec854b4e8e62658f5722ec38', 'request_id': 'req-89be1bee-d0e4-420e-963e-ee9bfd70e05c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:50.230766 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42448e35ec854b4e8e62658f5722ec38', 'request_id': 'req-89be1bee-d0e4-420e-963e-ee9bfd70e05c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:50.256003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] The token cache is being invalidate because project 42448e35ec854b4e8e62658f5722ec38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:50.256442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:50.257821 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-89be1bee-d0e4-420e-963e-ee9bfd70e05c tempest-VolumeQuotaClassesTest-1541233770 tempest-VolumeQuotaClassesTest-1541233770-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:50.259710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1164/2298] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:50 2022] DELETE /identity/v3/projects/42448e35ec854b4e8e62658f5722ec38 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:50.711208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824c9cc-4bb8-4b27-bc26-0377bba9f841 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:50.778284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:50.897099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6824c9cc-4bb8-4b27-bc26-0377bba9f841 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:51.010168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:51.011113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:51.011492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:51.011868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] PATH_INFO: `/v3/users/49294a2924e944a7b5fcca32c2e8683b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:51.013185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=49294a2924e944a7b5fcca32c2e8683b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:51.015539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:51.105706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49294a2924e944a7b5fcca32c2e8683b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:51.121436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49294a2924e944a7b5fcca32c2e8683b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:51.144842 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49294a2924e944a7b5fcca32c2e8683b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:51.147122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49294a2924e944a7b5fcca32c2e8683b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:51.158586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49294a2924e944a7b5fcca32c2e8683b'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:51.168763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1135/2299] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:50 2022] DELETE /identity/v3/users/49294a2924e944a7b5fcca32c2e8683b => generated 0 bytes in 458 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:51.173242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b2fbb-5aa9-4794-831b-7cfb3fe847f1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:51.174067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b2fbb-5aa9-4794-831b-7cfb3fe847f1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:51.176946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52b2fbb-5aa9-4794-831b-7cfb3fe847f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:51.209471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c52b2fbb-5aa9-4794-831b-7cfb3fe847f1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:51.211819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c52b2fbb-5aa9-4794-831b-7cfb3fe847f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:51.298705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:51.418694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1165/2300] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:51.429269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:51.444024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:51.554372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:51.575321 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56aec760-6f55-4039-aa9b-f250525cb8cd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:51.594730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5992dd-347a-4139-8f40-f753d4e1de25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:51.595393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5992dd-347a-4139-8f40-f753d4e1de25 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:51.595678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5992dd-347a-4139-8f40-f753d4e1de25 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:51.595946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5992dd-347a-4139-8f40-f753d4e1de25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:51.615740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5992dd-347a-4139-8f40-f753d4e1de25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:51.617498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5992dd-347a-4139-8f40-f753d4e1de25 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:51.630347 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1136/2301] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:52.275014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b2fbb-5aa9-4794-831b-7cfb3fe847f1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:52.279498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c52b2fbb-5aa9-4794-831b-7cfb3fe847f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:52.313821 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52b2fbb-5aa9-4794-831b-7cfb3fe847f1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:52.329724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:52.330675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:52.331158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:52.331919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] PATH_INFO: `/v3/projects/6dfbc1dfc78f437b8722404536e8c5fd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:52.334414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dfbc1dfc78f437b8722404536e8c5fd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:52.340016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:52.464834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dfbc1dfc78f437b8722404536e8c5fd', 'request_id': 'req-6f663367-4d44-4a63-bda5-8f04a05e5e31'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.473916 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dfbc1dfc78f437b8722404536e8c5fd', 'request_id': 'req-6f663367-4d44-4a63-bda5-8f04a05e5e31'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.482492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dfbc1dfc78f437b8722404536e8c5fd', 'request_id': 'req-6f663367-4d44-4a63-bda5-8f04a05e5e31'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.512529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] The token cache is being invalidate because project 6dfbc1dfc78f437b8722404536e8c5fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:52.515150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.518036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.521550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1166/2302] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:52 2022] DELETE /identity/v3/projects/6dfbc1dfc78f437b8722404536e8c5fd => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:52.525984 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5992dd-347a-4139-8f40-f753d4e1de25 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:52.547622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5992dd-347a-4139-8f40-f753d4e1de25 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:52.711751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:52.712405 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:52.712851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:52.713367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] PATH_INFO: `/v3/users/a3335b5fac2e4a9aa1260a88b0a94543` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:52.715005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a3335b5fac2e4a9aa1260a88b0a94543)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:52.717795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:52.806144 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a3335b5fac2e4a9aa1260a88b0a94543'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.817268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a3335b5fac2e4a9aa1260a88b0a94543'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.836611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a3335b5fac2e4a9aa1260a88b0a94543'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.837769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a3335b5fac2e4a9aa1260a88b0a94543'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.852221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f5abfa1a-0b0c-45af-a565-6a27cbc01197 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a3335b5fac2e4a9aa1260a88b0a94543'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:52.859702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1137/2303] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:02:52 2022] DELETE /identity/v3/users/a3335b5fac2e4a9aa1260a88b0a94543 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:53.225136 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:53.227161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:53.245793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f663367-4d44-4a63-bda5-8f04a05e5e31 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:53.343066 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:53.343799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:53.344095 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:53.344478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] PATH_INFO: `/v3/projects/1e85c01c0f054a3a90c46870f96cffe3` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:53.345503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e85c01c0f054a3a90c46870f96cffe3)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:53.347464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:53.412172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e85c01c0f054a3a90c46870f96cffe3', 'request_id': 'req-7b1668c7-31d6-477d-86f9-ba8a774d6c63'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:53.419303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e85c01c0f054a3a90c46870f96cffe3', 'request_id': 'req-7b1668c7-31d6-477d-86f9-ba8a774d6c63'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:53.421305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:53.429192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e85c01c0f054a3a90c46870f96cffe3', 'request_id': 'req-7b1668c7-31d6-477d-86f9-ba8a774d6c63'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:53.452529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] The token cache is being invalidate because project 1e85c01c0f054a3a90c46870f96cffe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:02:53.452876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:53.453561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-7b1668c7-31d6-477d-86f9-ba8a774d6c63 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:02:53.459247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1167/2304] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:02:53 2022] DELETE /identity/v3/projects/1e85c01c0f054a3a90c46870f96cffe3 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:53.460636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7a038-49d6-4bca-b9cb-ca5af891f64c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:53.461014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7a038-49d6-4bca-b9cb-ca5af891f64c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:53.461413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac7a038-49d6-4bca-b9cb-ca5af891f64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:53.486069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5ac7a038-49d6-4bca-b9cb-ca5af891f64c None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:53.488074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5ac7a038-49d6-4bca-b9cb-ca5af891f64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:53.603836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1138/2305] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:53.611481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eba6ff-ece4-4238-bc87-82f77c88c616 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:53.611770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eba6ff-ece4-4238-bc87-82f77c88c616 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:53.612048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eba6ff-ece4-4238-bc87-82f77c88c616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:53.647658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c6eba6ff-ece4-4238-bc87-82f77c88c616 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:53.649402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c6eba6ff-ece4-4238-bc87-82f77c88c616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:53.674539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1168/2306] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:53.683572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7a038-49d6-4bca-b9cb-ca5af891f64c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:53.706095 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac7a038-49d6-4bca-b9cb-ca5af891f64c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:53.724845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799c421-c2eb-47ff-9f13-b04b0b8c84b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:53.725696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799c421-c2eb-47ff-9f13-b04b0b8c84b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:53.727363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799c421-c2eb-47ff-9f13-b04b0b8c84b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:53.728074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799c421-c2eb-47ff-9f13-b04b0b8c84b9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:53.731310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799c421-c2eb-47ff-9f13-b04b0b8c84b9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:02:53.732983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799c421-c2eb-47ff-9f13-b04b0b8c84b9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:53.738479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799c421-c2eb-47ff-9f13-b04b0b8c84b9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:53.744423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:53.768907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1139/2307] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:02:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:02:53.769383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eba6ff-ece4-4238-bc87-82f77c88c616 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:53.782895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eba6ff-ece4-4238-bc87-82f77c88c616 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:53.795851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07c4dd-dab2-49a0-b226-e582bbae2cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:53.796824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07c4dd-dab2-49a0-b226-e582bbae2cb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:53.797279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07c4dd-dab2-49a0-b226-e582bbae2cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:53.803416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c07c4dd-dab2-49a0-b226-e582bbae2cb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:53.804621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07c4dd-dab2-49a0-b226-e582bbae2cb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:53.808035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c07c4dd-dab2-49a0-b226-e582bbae2cb2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:53.893193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1169/2308] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:53 2022] POST /identity/v3/projects => generated 379 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:02:53.895275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799c421-c2eb-47ff-9f13-b04b0b8c84b9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:53.912171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799c421-c2eb-47ff-9f13-b04b0b8c84b9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:53.989032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65b439-cd02-4c0c-a901-df991e63863e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:53.989939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65b439-cd02-4c0c-a901-df991e63863e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:53.990444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65b439-cd02-4c0c-a901-df991e63863e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:53.990913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f65b439-cd02-4c0c-a901-df991e63863e admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:53.991746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65b439-cd02-4c0c-a901-df991e63863e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:53.994255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f65b439-cd02-4c0c-a901-df991e63863e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:54.028253 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1140/2309] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:53 2022] POST /identity/v3/users => generated 290 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:54.033323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07c4dd-dab2-49a0-b226-e582bbae2cb2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:54.051735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c07c4dd-dab2-49a0-b226-e582bbae2cb2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:54.058736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2faec0-0b53-41c2-a5aa-c3c593269e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:54.059586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2faec0-0b53-41c2-a5aa-c3c593269e53 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:54.059876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2faec0-0b53-41c2-a5aa-c3c593269e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:54.060150 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2faec0-0b53-41c2-a5aa-c3c593269e53 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:54.060803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2faec0-0b53-41c2-a5aa-c3c593269e53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:54.062524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2faec0-0b53-41c2-a5aa-c3c593269e53 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:54.078254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1170/2310] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:54.082005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65b439-cd02-4c0c-a901-df991e63863e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:54.094853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f65b439-cd02-4c0c-a901-df991e63863e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:54.100588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a06fe7-a810-4147-842c-ffcb8d9070ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:54.101310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a06fe7-a810-4147-842c-ffcb8d9070ae admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:54.101489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a06fe7-a810-4147-842c-ffcb8d9070ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:54.101960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a06fe7-a810-4147-842c-ffcb8d9070ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:54.102578 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a06fe7-a810-4147-842c-ffcb8d9070ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:54.104896 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a06fe7-a810-4147-842c-ffcb8d9070ae admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:54.121602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1141/2311] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:02:54.125514 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2faec0-0b53-41c2-a5aa-c3c593269e53 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:54.138451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2faec0-0b53-41c2-a5aa-c3c593269e53 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:54.160472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcffe54-f2a5-4cba-b3d8-5e03a94bb0a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:54.162455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcffe54-f2a5-4cba-b3d8-5e03a94bb0a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:54.163930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcffe54-f2a5-4cba-b3d8-5e03a94bb0a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:54.164385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcffe54-f2a5-4cba-b3d8-5e03a94bb0a8 admin admin] PATH_INFO: `/v3/projects/16223b6f84b04628971ce02aa1c28ae8/users/ee12a357c2924e36a8e7d7cd04002ee2/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:54.204797 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcffe54-f2a5-4cba-b3d8-5e03a94bb0a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16223b6f84b04628971ce02aa1c28ae8, user_id=ee12a357c2924e36a8e7d7cd04002ee2, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:54.217032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcffe54-f2a5-4cba-b3d8-5e03a94bb0a8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:54.247217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1171/2312] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:54 2022] PUT /identity/v3/projects/16223b6f84b04628971ce02aa1c28ae8/users/ee12a357c2924e36a8e7d7cd04002ee2/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:54.258508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec27bf-2383-4804-b6db-f72e03a3ac4b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:54.258798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec27bf-2383-4804-b6db-f72e03a3ac4b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:54.259071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec27bf-2383-4804-b6db-f72e03a3ac4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:54.327883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e1ec27bf-2383-4804-b6db-f72e03a3ac4b None None] MFA Rules not processed for user `ee12a357c2924e36a8e7d7cd04002ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:54.329654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e1ec27bf-2383-4804-b6db-f72e03a3ac4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:54.444188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1142/2313] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:54.449643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcffe54-f2a5-4cba-b3d8-5e03a94bb0a8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:54.537694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcffe54-f2a5-4cba-b3d8-5e03a94bb0a8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:54.543917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c9e27-ff78-40d9-a323-65c0593abc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:54.544560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c9e27-ff78-40d9-a323-65c0593abc22 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:54.544839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c9e27-ff78-40d9-a323-65c0593abc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:54.545117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696c9e27-ff78-40d9-a323-65c0593abc22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:54.545764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c9e27-ff78-40d9-a323-65c0593abc22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:54.547848 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696c9e27-ff78-40d9-a323-65c0593abc22 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:54.605745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1172/2314] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:54 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:54.609903 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec27bf-2383-4804-b6db-f72e03a3ac4b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:54.624928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec27bf-2383-4804-b6db-f72e03a3ac4b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:54.707059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b5e35-f4da-416a-be1e-b97b0dcd2fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:54.707679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b5e35-f4da-416a-be1e-b97b0dcd2fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:54.707960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b5e35-f4da-416a-be1e-b97b0dcd2fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:54.708236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7b5e35-f4da-416a-be1e-b97b0dcd2fc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:54.710375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b5e35-f4da-416a-be1e-b97b0dcd2fc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:54.712467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7b5e35-f4da-416a-be1e-b97b0dcd2fc9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:54.724201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:54.765998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1143/2315] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:54 2022] POST /identity/v3/users => generated 289 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:54.767138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c9e27-ff78-40d9-a323-65c0593abc22 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:54.785967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696c9e27-ff78-40d9-a323-65c0593abc22 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:54.801624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389da1f-ec46-4291-bb57-7eb6a8f73959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:54.802695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389da1f-ec46-4291-bb57-7eb6a8f73959 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:54.803147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389da1f-ec46-4291-bb57-7eb6a8f73959 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:54.803781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4389da1f-ec46-4291-bb57-7eb6a8f73959 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:54.806171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389da1f-ec46-4291-bb57-7eb6a8f73959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:54.808888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4389da1f-ec46-4291-bb57-7eb6a8f73959 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:54.827132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1173/2316] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:54.830947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b5e35-f4da-416a-be1e-b97b0dcd2fc9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:54.862182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7b5e35-f4da-416a-be1e-b97b0dcd2fc9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:54.876616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712231e2-7ba2-40fe-9aa8-6afa70b2e1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:54.877730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712231e2-7ba2-40fe-9aa8-6afa70b2e1e5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:54.878063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712231e2-7ba2-40fe-9aa8-6afa70b2e1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:54.878489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712231e2-7ba2-40fe-9aa8-6afa70b2e1e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:54.879162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712231e2-7ba2-40fe-9aa8-6afa70b2e1e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:54.881988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712231e2-7ba2-40fe-9aa8-6afa70b2e1e5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:54.922988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1144/2317] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:54 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:54.926816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389da1f-ec46-4291-bb57-7eb6a8f73959 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:54.943953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4389da1f-ec46-4291-bb57-7eb6a8f73959 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:54.951338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c50ce2-7776-4abc-a416-cada7b39b4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:54.952200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c50ce2-7776-4abc-a416-cada7b39b4fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:54.952484 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c50ce2-7776-4abc-a416-cada7b39b4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:54.952761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c50ce2-7776-4abc-a416-cada7b39b4fe admin admin] PATH_INFO: `/v3/projects/5d3f2c7dc1e1442aa22b8b0500990414/users/8719bedc0b504c52912f4c5ddddfb0e7/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:55.003179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c50ce2-7776-4abc-a416-cada7b39b4fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d3f2c7dc1e1442aa22b8b0500990414, user_id=8719bedc0b504c52912f4c5ddddfb0e7, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:55.006355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c50ce2-7776-4abc-a416-cada7b39b4fe admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:55.034937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1174/2318] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:54 2022] PUT /identity/v3/projects/5d3f2c7dc1e1442aa22b8b0500990414/users/8719bedc0b504c52912f4c5ddddfb0e7/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:55.045995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba4989-d329-4070-90c2-9defe669dd6b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:55.046402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba4989-d329-4070-90c2-9defe669dd6b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:55.046770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba4989-d329-4070-90c2-9defe669dd6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:55.098689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-deba4989-d329-4070-90c2-9defe669dd6b None None] MFA Rules not processed for user `8719bedc0b504c52912f4c5ddddfb0e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:55.100237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-deba4989-d329-4070-90c2-9defe669dd6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:55.226256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1145/2319] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:55.231520 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c50ce2-7776-4abc-a416-cada7b39b4fe admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:55.269099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:55.312624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c50ce2-7776-4abc-a416-cada7b39b4fe admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:55.318317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34806dad-d537-4f7e-af63-4d1fb4a2be86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:55.319149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34806dad-d537-4f7e-af63-4d1fb4a2be86 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:55.319581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34806dad-d537-4f7e-af63-4d1fb4a2be86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:55.320058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34806dad-d537-4f7e-af63-4d1fb4a2be86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:55.320852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34806dad-d537-4f7e-af63-4d1fb4a2be86 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:55.323088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34806dad-d537-4f7e-af63-4d1fb4a2be86 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:55.363412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1175/2320] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:02:55 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:55.369878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba4989-d329-4070-90c2-9defe669dd6b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:55.383452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba4989-d329-4070-90c2-9defe669dd6b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:55.464276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4e4aa-0cfb-436f-8d54-86cdb379004e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:55.465058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4e4aa-0cfb-436f-8d54-86cdb379004e admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:55.465433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4e4aa-0cfb-436f-8d54-86cdb379004e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:55.465805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c4e4aa-0cfb-436f-8d54-86cdb379004e admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:55.466757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4e4aa-0cfb-436f-8d54-86cdb379004e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:55.469129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c4e4aa-0cfb-436f-8d54-86cdb379004e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:55.499941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1146/2321] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:02:55 2022] POST /identity/v3/users => generated 296 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:02:55.504512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34806dad-d537-4f7e-af63-4d1fb4a2be86 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:55.515799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34806dad-d537-4f7e-af63-4d1fb4a2be86 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:55.521829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f36a8-253d-46a4-ae81-716838d078b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:55.523270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f36a8-253d-46a4-ae81-716838d078b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:55.523764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f36a8-253d-46a4-ae81-716838d078b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:55.524160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648f36a8-253d-46a4-ae81-716838d078b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:55.524908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f36a8-253d-46a4-ae81-716838d078b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:55.527110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648f36a8-253d-46a4-ae81-716838d078b9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:55.544498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4e4aa-0cfb-436f-8d54-86cdb379004e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:55.545553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1176/2322] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:55.555004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c4e4aa-0cfb-436f-8d54-86cdb379004e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:55.560695 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b781e6-7b77-488b-8565-54d2fd055c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:55.561457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b781e6-7b77-488b-8565-54d2fd055c42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:55.561834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b781e6-7b77-488b-8565-54d2fd055c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:55.562530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b781e6-7b77-488b-8565-54d2fd055c42 admin admin] PATH_INFO: `/v3/projects/5f4ab3bb72534d24a40e426f1cb1571f/users/b05e2862fbe44376bb13d7f3c9b3ddb0/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:55.582750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b781e6-7b77-488b-8565-54d2fd055c42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4ab3bb72534d24a40e426f1cb1571f, user_id=b05e2862fbe44376bb13d7f3c9b3ddb0, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:55.585635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b781e6-7b77-488b-8565-54d2fd055c42 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:55.631288 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1147/2323] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:55 2022] PUT /identity/v3/projects/5f4ab3bb72534d24a40e426f1cb1571f/users/b05e2862fbe44376bb13d7f3c9b3ddb0/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:55.635738 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f36a8-253d-46a4-ae81-716838d078b9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:55.647165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648f36a8-253d-46a4-ae81-716838d078b9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:55.729705 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6d299-cde2-4b65-aba0-4ff2cc67d9ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:55.730485 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6d299-cde2-4b65-aba0-4ff2cc67d9ad admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:55.730837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6d299-cde2-4b65-aba0-4ff2cc67d9ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:55.731200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6d299-cde2-4b65-aba0-4ff2cc67d9ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:55.731925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6d299-cde2-4b65-aba0-4ff2cc67d9ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:55.733630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6d299-cde2-4b65-aba0-4ff2cc67d9ad admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:55.750582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b781e6-7b77-488b-8565-54d2fd055c42 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:55.752803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1177/2324] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:02:55 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:02:55.761859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b781e6-7b77-488b-8565-54d2fd055c42 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:55.766651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154dc9e4-9a14-423f-a224-f36d6d0771b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:55.767330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dc9e4-9a14-423f-a224-f36d6d0771b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:55.767609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dc9e4-9a14-423f-a224-f36d6d0771b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:55.767878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154dc9e4-9a14-423f-a224-f36d6d0771b1 admin admin] PATH_INFO: `/v3/projects/5f4ab3bb72534d24a40e426f1cb1571f/users/b05e2862fbe44376bb13d7f3c9b3ddb0/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:55.769794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154dc9e4-9a14-423f-a224-f36d6d0771b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f4ab3bb72534d24a40e426f1cb1571f, user_id=b05e2862fbe44376bb13d7f3c9b3ddb0, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:55.772687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154dc9e4-9a14-423f-a224-f36d6d0771b1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:55.784340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1148/2325] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:02:55 2022] PUT /identity/v3/projects/5f4ab3bb72534d24a40e426f1cb1571f/users/b05e2862fbe44376bb13d7f3c9b3ddb0/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:02:55.796654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac62cc-c1ef-41d7-9e34-08eb92705be8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:55.796654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac62cc-c1ef-41d7-9e34-08eb92705be8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:55.796843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac62cc-c1ef-41d7-9e34-08eb92705be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:55.835846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3bac62cc-c1ef-41d7-9e34-08eb92705be8 None None] MFA Rules not processed for user `b05e2862fbe44376bb13d7f3c9b3ddb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:55.837093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3bac62cc-c1ef-41d7-9e34-08eb92705be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:55.974490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1178/2326] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:55 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:55.991450 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154dc9e4-9a14-423f-a224-f36d6d0771b1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:56.001164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-154dc9e4-9a14-423f-a224-f36d6d0771b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:56.064985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=7,keystone:SELECT=93 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:56.144354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154dc9e4-9a14-423f-a224-f36d6d0771b1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:56.148190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6134f2-f55e-4e23-af26-08fdbd5ed9b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:56.148825 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6134f2-f55e-4e23-af26-08fdbd5ed9b1 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:56.149105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6134f2-f55e-4e23-af26-08fdbd5ed9b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:56.149383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6134f2-f55e-4e23-af26-08fdbd5ed9b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:56.178162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6134f2-f55e-4e23-af26-08fdbd5ed9b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:56.179683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6134f2-f55e-4e23-af26-08fdbd5ed9b1 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:56.214476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1149/2327] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:55 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:56.260875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac62cc-c1ef-41d7-9e34-08eb92705be8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:56.262468 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3bac62cc-c1ef-41d7-9e34-08eb92705be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:56.279544 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac62cc-c1ef-41d7-9e34-08eb92705be8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:56.284966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae6a72-0a7c-4ab6-b03e-6251483e51b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:56.285636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae6a72-0a7c-4ab6-b03e-6251483e51b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:56.285920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae6a72-0a7c-4ab6-b03e-6251483e51b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:56.286308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ae6a72-0a7c-4ab6-b03e-6251483e51b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:56.322167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:56.376850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae6a72-0a7c-4ab6-b03e-6251483e51b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:56.379730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ae6a72-0a7c-4ab6-b03e-6251483e51b2 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:56.391435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1179/2328] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:56.456540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298dbe0-9f94-49cb-822e-7ded55a4a507 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:56.457515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298dbe0-9f94-49cb-822e-7ded55a4a507 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:56.458145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6298dbe0-9f94-49cb-822e-7ded55a4a507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:56.491595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6298dbe0-9f94-49cb-822e-7ded55a4a507 None None] MFA Rules not processed for user `2e34c55dc58a44ada90803e5530ab447`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:56.492841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6298dbe0-9f94-49cb-822e-7ded55a4a507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:56.574283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1150/2329] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:02:56 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:56.584115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae6a72-0a7c-4ab6-b03e-6251483e51b2 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:56.585656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d9ae6a72-0a7c-4ab6-b03e-6251483e51b2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:56.682280 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ae6a72-0a7c-4ab6-b03e-6251483e51b2 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:56.686356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3505c8965c4a42e493a2f5916cab56dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:56.687233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:56.687670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:56.688090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:56.690347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:56.795863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:56.798602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:56.824378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1180/2330] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:02:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:02:57.025722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23ed49-5800-480a-8053-c59a8ec68489 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:57.026767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23ed49-5800-480a-8053-c59a8ec68489 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:57.027208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f23ed49-5800-480a-8053-c59a8ec68489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:57.048224 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5f23ed49-5800-480a-8053-c59a8ec68489 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:57.049561 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5f23ed49-5800-480a-8053-c59a8ec68489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:57.122259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1151/2331] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:02:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:57.170588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:57.172094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:57.239834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:02:57.255066 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd035f33-965c-43c6-9712-abb8438d3074 service glance] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:57.260211 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b90985-7468-4641-bcda-f14315d1520d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:57.260776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b90985-7468-4641-bcda-f14315d1520d service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:57.261056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b90985-7468-4641-bcda-f14315d1520d service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:57.261333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b90985-7468-4641-bcda-f14315d1520d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:57.270239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b90985-7468-4641-bcda-f14315d1520d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:57.271874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b90985-7468-4641-bcda-f14315d1520d service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:57.280838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1181/2332] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:02:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:02:59.666739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99a62d-f112-411c-9764-7b5569423376 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:59.667906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99a62d-f112-411c-9764-7b5569423376 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:59.668340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c99a62d-f112-411c-9764-7b5569423376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:59.697449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1c99a62d-f112-411c-9764-7b5569423376 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:02:59.699310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1c99a62d-f112-411c-9764-7b5569423376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:02:59.723158 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1152/2333] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:02:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:02:59.741055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b90985-7468-4641-bcda-f14315d1520d service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:02:59.755849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b90985-7468-4641-bcda-f14315d1520d service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:02:59.771339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182cf2c4-b22d-4a76-be15-fe68dda4649a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:02:59.773118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cf2c4-b22d-4a76-be15-fe68dda4649a service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:02:59.773383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cf2c4-b22d-4a76-be15-fe68dda4649a service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:02:59.773663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182cf2c4-b22d-4a76-be15-fe68dda4649a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:02:59.799023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182cf2c4-b22d-4a76-be15-fe68dda4649a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:02:59.800670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182cf2c4-b22d-4a76-be15-fe68dda4649a service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:02:59.823417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1182/2334] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:02:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:03:00.575725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99a62d-f112-411c-9764-7b5569423376 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:00.577289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1c99a62d-f112-411c-9764-7b5569423376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:00.793437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:00.834065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c99a62d-f112-411c-9764-7b5569423376 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:00.862837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:00.863705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:00.864093 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:00.864489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] PATH_INFO: `/v3/users/1389f526e0d5490f9ee485415b6b49d6` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:00.866521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1389f526e0d5490f9ee485415b6b49d6)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:00.873904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:00.986969 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1389f526e0d5490f9ee485415b6b49d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:01.004510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1389f526e0d5490f9ee485415b6b49d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:01.060102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1389f526e0d5490f9ee485415b6b49d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:01.065946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1389f526e0d5490f9ee485415b6b49d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:01.099983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1389f526e0d5490f9ee485415b6b49d6'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:01.125796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1153/2335] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:03:00 2022] DELETE /identity/v3/users/1389f526e0d5490f9ee485415b6b49d6 => generated 0 bytes in 547 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:01.127646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4d517-f8a4-4b5d-8e94-ff4e5c676d85 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:01.128181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4d517-f8a4-4b5d-8e94-ff4e5c676d85 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:01.128429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc4d517-f8a4-4b5d-8e94-ff4e5c676d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:01.195809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fbc4d517-f8a4-4b5d-8e94-ff4e5c676d85 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:01.197103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fbc4d517-f8a4-4b5d-8e94-ff4e5c676d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:01.486957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1183/2336] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:01.501996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:01.505397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:01.669525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ec024-a409-42f3-87bb-6727a753d81a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:01.681317 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98989-1036-4efe-a804-e58079769321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:01.682026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98989-1036-4efe-a804-e58079769321 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:01.682544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98989-1036-4efe-a804-e58079769321 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:01.683009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c98989-1036-4efe-a804-e58079769321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:01.716280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98989-1036-4efe-a804-e58079769321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:01.717937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c98989-1036-4efe-a804-e58079769321 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:01.754592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1154/2337] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:02.508410 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4d517-f8a4-4b5d-8e94-ff4e5c676d85 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:02.511968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fbc4d517-f8a4-4b5d-8e94-ff4e5c676d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:02.535109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc4d517-f8a4-4b5d-8e94-ff4e5c676d85 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:02.542228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:02.543159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:02.543589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:02.544013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] PATH_INFO: `/v3/projects/2da60689edf04a8f946ca07f9ae268b8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:02.545312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2da60689edf04a8f946ca07f9ae268b8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:02.547607 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:02.686985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2da60689edf04a8f946ca07f9ae268b8', 'request_id': 'req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:02.725427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2da60689edf04a8f946ca07f9ae268b8', 'request_id': 'req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:02.746515 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2da60689edf04a8f946ca07f9ae268b8', 'request_id': 'req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:02.774613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] The token cache is being invalidate because project 2da60689edf04a8f946ca07f9ae268b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:03:02.775088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:02.776773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2904628c-ccf0-4e6c-9a14-d8e21ae0e9b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:02.784059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1184/2338] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:03:02 2022] DELETE /identity/v3/projects/2da60689edf04a8f946ca07f9ae268b8 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:03.219231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98989-1036-4efe-a804-e58079769321 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:03.222412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-54c98989-1036-4efe-a804-e58079769321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:03.240472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c98989-1036-4efe-a804-e58079769321 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:03.372145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:03.373962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:03.374720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:03.378661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] PATH_INFO: `/v3/users/ee12a357c2924e36a8e7d7cd04002ee2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:03.380521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee12a357c2924e36a8e7d7cd04002ee2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:03.384159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:03.400587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:03.498502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee12a357c2924e36a8e7d7cd04002ee2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:03.521033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee12a357c2924e36a8e7d7cd04002ee2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:03.551563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee12a357c2924e36a8e7d7cd04002ee2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:03.554888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee12a357c2924e36a8e7d7cd04002ee2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:03.596212 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee12a357c2924e36a8e7d7cd04002ee2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:03.627055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1155/2339] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:03:03 2022] DELETE /identity/v3/users/ee12a357c2924e36a8e7d7cd04002ee2 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:03.632057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ad9a-bb2b-4054-8bdb-f907fc564f22 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:03.632992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ad9a-bb2b-4054-8bdb-f907fc564f22 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:03.633590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5193ad9a-bb2b-4054-8bdb-f907fc564f22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:03.690411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5193ad9a-bb2b-4054-8bdb-f907fc564f22 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:03.693820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5193ad9a-bb2b-4054-8bdb-f907fc564f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:03.851127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:03.886532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1185/2340] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:03:03.894793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:03.897325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:04.037331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bca09ba-ac56-498b-bedf-c8ffa8535925 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:04.064716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a4df2-38f0-4f81-a947-cced4f5ac280 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:04.065770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a4df2-38f0-4f81-a947-cced4f5ac280 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:04.066235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a4df2-38f0-4f81-a947-cced4f5ac280 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:04.066605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68a4df2-38f0-4f81-a947-cced4f5ac280 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:04.100738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a4df2-38f0-4f81-a947-cced4f5ac280 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:04.104474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68a4df2-38f0-4f81-a947-cced4f5ac280 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:04.140455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1156/2341] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:04.603451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193ad9a-bb2b-4054-8bdb-f907fc564f22 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:04.604943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5193ad9a-bb2b-4054-8bdb-f907fc564f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:04.631450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5193ad9a-bb2b-4054-8bdb-f907fc564f22 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:04.650693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:04.652212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:04.652707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:04.653139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] PATH_INFO: `/v3/projects/16223b6f84b04628971ce02aa1c28ae8` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:04.656384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16223b6f84b04628971ce02aa1c28ae8)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:04.660566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:04.769912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16223b6f84b04628971ce02aa1c28ae8', 'request_id': 'req-c8d4bf25-6db7-4580-8387-217ca2078313'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:04.781884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16223b6f84b04628971ce02aa1c28ae8', 'request_id': 'req-c8d4bf25-6db7-4580-8387-217ca2078313'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:04.792111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16223b6f84b04628971ce02aa1c28ae8', 'request_id': 'req-c8d4bf25-6db7-4580-8387-217ca2078313'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:04.829251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] The token cache is being invalidate because project 16223b6f84b04628971ce02aa1c28ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:03:04.829915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:04.831865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:04.858967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a4df2-38f0-4f81-a947-cced4f5ac280 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:04.873396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68a4df2-38f0-4f81-a947-cced4f5ac280 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:04.875538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1186/2342] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:03:04 2022] DELETE /identity/v3/projects/16223b6f84b04628971ce02aa1c28ae8 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:05.104926 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:05.106489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:05.106489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:05.106489 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] PATH_INFO: `/v3/users/8719bedc0b504c52912f4c5ddddfb0e7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:05.107810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8719bedc0b504c52912f4c5ddddfb0e7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:05.109840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:05.241984 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:05.271848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8719bedc0b504c52912f4c5ddddfb0e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:05.331210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8719bedc0b504c52912f4c5ddddfb0e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:05.365499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8719bedc0b504c52912f4c5ddddfb0e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:05.366645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8719bedc0b504c52912f4c5ddddfb0e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:05.376376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8719bedc0b504c52912f4c5ddddfb0e7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:05.387181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1157/2343] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:03:04 2022] DELETE /identity/v3/users/8719bedc0b504c52912f4c5ddddfb0e7 => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:05.897840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:05.900898 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:05.928244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4bf25-6db7-4580-8387-217ca2078313 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:06.049435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:06.050122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:06.050413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:06.050866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] PATH_INFO: `/v3/projects/5d3f2c7dc1e1442aa22b8b0500990414` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:06.052161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d3f2c7dc1e1442aa22b8b0500990414)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:06.054295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:06.175277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d3f2c7dc1e1442aa22b8b0500990414', 'request_id': 'req-ae38eb04-0244-4031-bc16-519bf988ece0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.183937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d3f2c7dc1e1442aa22b8b0500990414', 'request_id': 'req-ae38eb04-0244-4031-bc16-519bf988ece0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.191509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d3f2c7dc1e1442aa22b8b0500990414', 'request_id': 'req-ae38eb04-0244-4031-bc16-519bf988ece0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.220276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] The token cache is being invalidate because project 5d3f2c7dc1e1442aa22b8b0500990414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:03:06.220674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.221427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.223280 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1187/2344] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:03:05 2022] DELETE /identity/v3/projects/5d3f2c7dc1e1442aa22b8b0500990414 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:06.227244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:06.229574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:06.246341 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7f5601-2ae1-44af-8bf8-f2cfdea76a5d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:06.370280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:06.371073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:06.371431 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:06.371796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] PATH_INFO: `/v3/users/b05e2862fbe44376bb13d7f3c9b3ddb0` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:06.375583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b05e2862fbe44376bb13d7f3c9b3ddb0)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:06.380483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:06.495052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b05e2862fbe44376bb13d7f3c9b3ddb0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.510255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b05e2862fbe44376bb13d7f3c9b3ddb0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.539951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b05e2862fbe44376bb13d7f3c9b3ddb0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.541597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b05e2862fbe44376bb13d7f3c9b3ddb0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.553183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bae32c23-c77a-45cf-94c7-7e83fae10a44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b05e2862fbe44376bb13d7f3c9b3ddb0'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:06.577543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1158/2345] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:03:06 2022] DELETE /identity/v3/users/b05e2862fbe44376bb13d7f3c9b3ddb0 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:07.131430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:07.132902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:07.169725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae38eb04-0244-4031-bc16-519bf988ece0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:07.180253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:07.274659 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:07.276800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:07.280484 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:07.281032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] PATH_INFO: `/v3/projects/5f4ab3bb72534d24a40e426f1cb1571f` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:07.282677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f4ab3bb72534d24a40e426f1cb1571f)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:07.284817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:07.375342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f4ab3bb72534d24a40e426f1cb1571f', 'request_id': 'req-52e6c104-c73d-4c74-aa3b-34fea9935a48'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:07.394684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f4ab3bb72534d24a40e426f1cb1571f', 'request_id': 'req-52e6c104-c73d-4c74-aa3b-34fea9935a48'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:07.410730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f4ab3bb72534d24a40e426f1cb1571f', 'request_id': 'req-52e6c104-c73d-4c74-aa3b-34fea9935a48'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:07.459670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] The token cache is being invalidate because project 5f4ab3bb72534d24a40e426f1cb1571f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:03:07.460421 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:07.461348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-52e6c104-c73d-4c74-aa3b-34fea9935a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:07.468952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1188/2346] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:03:07 2022] DELETE /identity/v3/projects/5f4ab3bb72534d24a40e426f1cb1571f => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:07.476674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd93959-31f3-423e-8fdd-1aff21c9622e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:07.480579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd93959-31f3-423e-8fdd-1aff21c9622e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:07.481689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd93959-31f3-423e-8fdd-1aff21c9622e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:07.529381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-8bd93959-31f3-423e-8fdd-1aff21c9622e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:07.530997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8bd93959-31f3-423e-8fdd-1aff21c9622e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:07.634609 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:07.726365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1159/2347] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:07.735502 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb33f9f-9767-4a96-8c9c-c9547cfab371 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:07.735850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb33f9f-9767-4a96-8c9c-c9547cfab371 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:07.736050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb33f9f-9767-4a96-8c9c-c9547cfab371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:07.782783 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ccb33f9f-9767-4a96-8c9c-c9547cfab371 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:07.785563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ccb33f9f-9767-4a96-8c9c-c9547cfab371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:07.810569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1189/2348] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:07.814577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd93959-31f3-423e-8fdd-1aff21c9622e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:07.829389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd93959-31f3-423e-8fdd-1aff21c9622e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:07.844089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08814e-584e-43a0-a903-82cc7b733233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:07.844089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08814e-584e-43a0-a903-82cc7b733233 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:07.844089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08814e-584e-43a0-a903-82cc7b733233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:07.844089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd08814e-584e-43a0-a903-82cc7b733233 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:07.844625 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08814e-584e-43a0-a903-82cc7b733233 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:03:07.844625 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08814e-584e-43a0-a903-82cc7b733233 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:07.850353 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd08814e-584e-43a0-a903-82cc7b733233 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:07.880717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1160/2349] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:03:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:03:07.885202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb33f9f-9767-4a96-8c9c-c9547cfab371 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:07.898221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb33f9f-9767-4a96-8c9c-c9547cfab371 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:07.911317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e9035-9ef4-4bc0-a83d-bdb36f7319df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:07.912179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e9035-9ef4-4bc0-a83d-bdb36f7319df admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:07.912464 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e9035-9ef4-4bc0-a83d-bdb36f7319df admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:07.912715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4e9035-9ef4-4bc0-a83d-bdb36f7319df admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:07.913768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e9035-9ef4-4bc0-a83d-bdb36f7319df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:07.915346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4e9035-9ef4-4bc0-a83d-bdb36f7319df admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:07.975799 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1190/2350] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:03:07 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:07.979774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08814e-584e-43a0-a903-82cc7b733233 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:07.995912 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd08814e-584e-43a0-a903-82cc7b733233 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:08.128000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72470d34-4881-48f8-b17f-899d62f3e27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:08.129278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72470d34-4881-48f8-b17f-899d62f3e27f admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:08.129803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72470d34-4881-48f8-b17f-899d62f3e27f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:08.130313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72470d34-4881-48f8-b17f-899d62f3e27f admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:08.131190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72470d34-4881-48f8-b17f-899d62f3e27f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:08.133483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72470d34-4881-48f8-b17f-899d62f3e27f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:08.202066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1161/2351] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:03:07 2022] POST /identity/v3/users => generated 287 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:08.206446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e9035-9ef4-4bc0-a83d-bdb36f7319df admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:08.224029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4e9035-9ef4-4bc0-a83d-bdb36f7319df admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:08.230425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274d5da-bc6b-4d31-8b58-ab9c13db9ffc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:08.231146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274d5da-bc6b-4d31-8b58-ab9c13db9ffc admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:08.231491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274d5da-bc6b-4d31-8b58-ab9c13db9ffc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:08.232252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1274d5da-bc6b-4d31-8b58-ab9c13db9ffc admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:08.232990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274d5da-bc6b-4d31-8b58-ab9c13db9ffc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:08.234825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1274d5da-bc6b-4d31-8b58-ab9c13db9ffc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:08.247255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1191/2352] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:03:08.251288 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72470d34-4881-48f8-b17f-899d62f3e27f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:08.269749 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72470d34-4881-48f8-b17f-899d62f3e27f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:08.277014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551ba184-7781-4c25-8f8e-be35c09d3e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:08.278289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba184-7781-4c25-8f8e-be35c09d3e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:08.278808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba184-7781-4c25-8f8e-be35c09d3e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:08.279615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551ba184-7781-4c25-8f8e-be35c09d3e2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:08.280532 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ba184-7781-4c25-8f8e-be35c09d3e2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:08.282820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551ba184-7781-4c25-8f8e-be35c09d3e2f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:08.299810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1162/2353] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:08 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:08.303644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274d5da-bc6b-4d31-8b58-ab9c13db9ffc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:08.316362 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1274d5da-bc6b-4d31-8b58-ab9c13db9ffc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:08.323166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac20e04-9567-492e-bae1-3c1d56418ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:08.324491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac20e04-9567-492e-bae1-3c1d56418ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:08.324988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac20e04-9567-492e-bae1-3c1d56418ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:08.325675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac20e04-9567-492e-bae1-3c1d56418ec7 admin admin] PATH_INFO: `/v3/projects/f061702879f74ee1a6d1bb7722370cbc/users/51e461eb2801482d829cdc66eeb61f1c/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:08.368558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac20e04-9567-492e-bae1-3c1d56418ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f061702879f74ee1a6d1bb7722370cbc, user_id=51e461eb2801482d829cdc66eeb61f1c, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:08.372441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac20e04-9567-492e-bae1-3c1d56418ec7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:08.397043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1192/2354] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:03:08 2022] PUT /identity/v3/projects/f061702879f74ee1a6d1bb7722370cbc/users/51e461eb2801482d829cdc66eeb61f1c/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:08.402633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e052b5a-3aac-42af-b2c6-19d8be09ba7f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:08.403383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e052b5a-3aac-42af-b2c6-19d8be09ba7f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:08.403779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e052b5a-3aac-42af-b2c6-19d8be09ba7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:08.440448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-1e052b5a-3aac-42af-b2c6-19d8be09ba7f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:08.462781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1e052b5a-3aac-42af-b2c6-19d8be09ba7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:08.840099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1163/2355] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:08.850323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac20e04-9567-492e-bae1-3c1d56418ec7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:08.852559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9ac20e04-9567-492e-bae1-3c1d56418ec7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:08.947171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:08.974760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac20e04-9567-492e-bae1-3c1d56418ec7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:09.008923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecf3a23-6f20-473a-aed1-dec25ed0c503 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:09.008923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf3a23-6f20-473a-aed1-dec25ed0c503 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:09.008923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf3a23-6f20-473a-aed1-dec25ed0c503 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:09.009455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecf3a23-6f20-473a-aed1-dec25ed0c503 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:09.054940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf3a23-6f20-473a-aed1-dec25ed0c503 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:09.057111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eecf3a23-6f20-473a-aed1-dec25ed0c503 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:09.080613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1193/2356] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:10.821143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7051-c37f-413d-9a70-c2bafcdd46b8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:10.822611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7051-c37f-413d-9a70-c2bafcdd46b8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:10.823145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98f7051-c37f-413d-9a70-c2bafcdd46b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:10.826666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0084d1c-b4cb-4b66-a584-76a693897118 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:10.827197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0084d1c-b4cb-4b66-a584-76a693897118 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:10.827410 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0084d1c-b4cb-4b66-a584-76a693897118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:10.876201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a0084d1c-b4cb-4b66-a584-76a693897118 None None] MFA Rules not processed for user `5a0fd7347cad41bc80f427f872cf78fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:10.880018 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a0084d1c-b4cb-4b66-a584-76a693897118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:10.894546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b98f7051-c37f-413d-9a70-c2bafcdd46b8 None None] MFA Rules not processed for user `51e461eb2801482d829cdc66eeb61f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:10.896248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b98f7051-c37f-413d-9a70-c2bafcdd46b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:10.961887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:11.039517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1194/2357] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:11.117440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1164/2358] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:11.135806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0084d1c-b4cb-4b66-a584-76a693897118 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:11.158763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0084d1c-b4cb-4b66-a584-76a693897118 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:11.164138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fb287-6f06-4550-9766-c9bf89d235ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:11.165016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fb287-6f06-4550-9766-c9bf89d235ac admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:11.165474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fb287-6f06-4550-9766-c9bf89d235ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:11.166016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1fb287-6f06-4550-9766-c9bf89d235ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:11.166977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fb287-6f06-4550-9766-c9bf89d235ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:11.169198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1fb287-6f06-4550-9766-c9bf89d235ac admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:11.222903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1195/2359] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:03:11 2022] POST /identity/v3/projects => generated 373 bytes in 85 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:03:11.224070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f7051-c37f-413d-9a70-c2bafcdd46b8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:11.236639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98f7051-c37f-413d-9a70-c2bafcdd46b8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:11.320151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be23e24-1062-4009-9b01-8b1aef39cf00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:11.320870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be23e24-1062-4009-9b01-8b1aef39cf00 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:11.321233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be23e24-1062-4009-9b01-8b1aef39cf00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:11.321369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be23e24-1062-4009-9b01-8b1aef39cf00 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:11.322119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be23e24-1062-4009-9b01-8b1aef39cf00 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:11.324051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be23e24-1062-4009-9b01-8b1aef39cf00 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:11.370957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1165/2360] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:03:11 2022] POST /identity/v3/users => generated 293 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:11.372273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fb287-6f06-4550-9766-c9bf89d235ac admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:11.387486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1fb287-6f06-4550-9766-c9bf89d235ac admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:11.400783 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c64af8-ecee-44de-9540-9c50404c8507 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:11.401718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c64af8-ecee-44de-9540-9c50404c8507 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:11.401718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c64af8-ecee-44de-9540-9c50404c8507 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:11.402006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c64af8-ecee-44de-9540-9c50404c8507 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:11.402652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c64af8-ecee-44de-9540-9c50404c8507 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:11.406853 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c64af8-ecee-44de-9540-9c50404c8507 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:11.433064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1196/2361] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:11.434132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be23e24-1062-4009-9b01-8b1aef39cf00 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:11.445488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be23e24-1062-4009-9b01-8b1aef39cf00 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:11.453646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db929f68-ab38-4cbc-960f-d0f55467b47d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:11.458382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db929f68-ab38-4cbc-960f-d0f55467b47d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:11.458680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db929f68-ab38-4cbc-960f-d0f55467b47d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:11.458966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db929f68-ab38-4cbc-960f-d0f55467b47d admin admin] PATH_INFO: `/v3/projects/fdb4f665127d484192cac83dd1069cc7/users/85b5ab4fa4534e00b5deef113ac48523/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:11.484244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db929f68-ab38-4cbc-960f-d0f55467b47d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb4f665127d484192cac83dd1069cc7, user_id=85b5ab4fa4534e00b5deef113ac48523, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:11.486989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db929f68-ab38-4cbc-960f-d0f55467b47d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:11.521018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1166/2362] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:03:11 2022] PUT /identity/v3/projects/fdb4f665127d484192cac83dd1069cc7/users/85b5ab4fa4534e00b5deef113ac48523/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:03:11.524824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c64af8-ecee-44de-9540-9c50404c8507 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:11.535527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c64af8-ecee-44de-9540-9c50404c8507 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:11.689453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2143c50d-ca6c-4a00-8b33-4ec1cd0e03e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:11.689987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143c50d-ca6c-4a00-8b33-4ec1cd0e03e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:11.690411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143c50d-ca6c-4a00-8b33-4ec1cd0e03e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:11.690596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2143c50d-ca6c-4a00-8b33-4ec1cd0e03e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:11.691238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143c50d-ca6c-4a00-8b33-4ec1cd0e03e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:11.692874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2143c50d-ca6c-4a00-8b33-4ec1cd0e03e1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:11.704068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1197/2363] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:11 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:11.707984 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db929f68-ab38-4cbc-960f-d0f55467b47d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:11.720510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db929f68-ab38-4cbc-960f-d0f55467b47d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:11.725802 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19423c9-9c28-4d0a-a3a3-1366ae12fb76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:11.726522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19423c9-9c28-4d0a-a3a3-1366ae12fb76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:11.726930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19423c9-9c28-4d0a-a3a3-1366ae12fb76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:11.727208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19423c9-9c28-4d0a-a3a3-1366ae12fb76 admin admin] PATH_INFO: `/v3/projects/fdb4f665127d484192cac83dd1069cc7/users/85b5ab4fa4534e00b5deef113ac48523/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:11.729395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19423c9-9c28-4d0a-a3a3-1366ae12fb76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb4f665127d484192cac83dd1069cc7, user_id=85b5ab4fa4534e00b5deef113ac48523, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:11.732475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19423c9-9c28-4d0a-a3a3-1366ae12fb76 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:11.746549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1167/2364] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:03:11 2022] PUT /identity/v3/projects/fdb4f665127d484192cac83dd1069cc7/users/85b5ab4fa4534e00b5deef113ac48523/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:12.748442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aff07c-da27-4b51-ba47-e6667699735e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:12.748970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aff07c-da27-4b51-ba47-e6667699735e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:12.749225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60aff07c-da27-4b51-ba47-e6667699735e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:12.828432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-60aff07c-da27-4b51-ba47-e6667699735e None None] MFA Rules not processed for user `85b5ab4fa4534e00b5deef113ac48523`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:12.831995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-60aff07c-da27-4b51-ba47-e6667699735e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:12.904250 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:12.975125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1198/2365] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:12.990275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19423c9-9c28-4d0a-a3a3-1366ae12fb76 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:12.992487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a19423c9-9c28-4d0a-a3a3-1366ae12fb76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:13.088148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19423c9-9c28-4d0a-a3a3-1366ae12fb76 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:13.092806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede1560-f320-43f5-a82a-d5969c9a7960 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:13.094033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede1560-f320-43f5-a82a-d5969c9a7960 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:13.094645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede1560-f320-43f5-a82a-d5969c9a7960 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:13.095123 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ede1560-f320-43f5-a82a-d5969c9a7960 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:13.141944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:13.202349 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede1560-f320-43f5-a82a-d5969c9a7960 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:13.204378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ede1560-f320-43f5-a82a-d5969c9a7960 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:13.231483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda529b-f829-4252-a5d8-fdbe75f7d8a0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:13.231832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda529b-f829-4252-a5d8-fdbe75f7d8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:13.232143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dda529b-f829-4252-a5d8-fdbe75f7d8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:13.254768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1168/2366] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:13.294669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5dda529b-f829-4252-a5d8-fdbe75f7d8a0 None None] MFA Rules not processed for user `baa15cc84259470f93afcfe23ef55c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:13.296019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5dda529b-f829-4252-a5d8-fdbe75f7d8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:13.483770 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1199/2367] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:13 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:15.029715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede1560-f320-43f5-a82a-d5969c9a7960 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:15.043740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ede1560-f320-43f5-a82a-d5969c9a7960 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:15.067739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f93ee6-cd03-4a10-9ae4-bbcadb3d248e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:15.069557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f93ee6-cd03-4a10-9ae4-bbcadb3d248e service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:15.069857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f93ee6-cd03-4a10-9ae4-bbcadb3d248e service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:15.070623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f93ee6-cd03-4a10-9ae4-bbcadb3d248e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:15.106738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f93ee6-cd03-4a10-9ae4-bbcadb3d248e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:15.108350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f93ee6-cd03-4a10-9ae4-bbcadb3d248e service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:15.131607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1169/2368] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:15 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:23.465516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=37 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:25.115924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=18 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:26.412398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93c06-d0c0-4a6e-b8df-4d6c99f68290 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:26.412861 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93c06-d0c0-4a6e-b8df-4d6c99f68290 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:26.412954 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b93c06-d0c0-4a6e-b8df-4d6c99f68290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:26.451104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d3b93c06-d0c0-4a6e-b8df-4d6c99f68290 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:26.452622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d3b93c06-d0c0-4a6e-b8df-4d6c99f68290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:26.535665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1200/2369] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:03:26 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:26.569439 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f93ee6-cd03-4a10-9ae4-bbcadb3d248e service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:26.571088 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a4f93ee6-cd03-4a10-9ae4-bbcadb3d248e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:26.695657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f93ee6-cd03-4a10-9ae4-bbcadb3d248e service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:26.702122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f97b7ff-3272-466f-94af-4e6ef970643a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:26.702957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97b7ff-3272-466f-94af-4e6ef970643a service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:26.703781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97b7ff-3272-466f-94af-4e6ef970643a service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:26.704126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f97b7ff-3272-466f-94af-4e6ef970643a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:26.714108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97b7ff-3272-466f-94af-4e6ef970643a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:26.715960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f97b7ff-3272-466f-94af-4e6ef970643a service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:26.724418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1170/2370] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:03:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:03:26.912101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a69ee-6fc5-435d-abe6-ef5c01e650e1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:26.912391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a69ee-6fc5-435d-abe6-ef5c01e650e1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:26.912669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a69ee-6fc5-435d-abe6-ef5c01e650e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:26.940302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b51a69ee-6fc5-435d-abe6-ef5c01e650e1 None None] MFA Rules not processed for user `0d8d0ff010c04d65b36ebe65c1cbbbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:26.941507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b51a69ee-6fc5-435d-abe6-ef5c01e650e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:27.004076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1201/2371] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:30.341148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec009c0d-f833-4798-858c-f8195f56f396 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:30.341803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec009c0d-f833-4798-858c-f8195f56f396 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:30.341803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec009c0d-f833-4798-858c-f8195f56f396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:30.369712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ec009c0d-f833-4798-858c-f8195f56f396 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:30.372110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ec009c0d-f833-4798-858c-f8195f56f396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:30.390226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1171/2372] 10.222.0.25 () {58 vars in 967 bytes} [Tue Sep 13 07:03:30 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:30.437036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a69ee-6fc5-435d-abe6-ef5c01e650e1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:30.454168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a69ee-6fc5-435d-abe6-ef5c01e650e1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:30.481745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9aa23ef-79b1-48e9-8cba-ac73a4cc7298 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:30.490480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa23ef-79b1-48e9-8cba-ac73a4cc7298 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:30.491037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa23ef-79b1-48e9-8cba-ac73a4cc7298 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:30.491608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9aa23ef-79b1-48e9-8cba-ac73a4cc7298 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:30.518392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa23ef-79b1-48e9-8cba-ac73a4cc7298 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:30.520768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9aa23ef-79b1-48e9-8cba-ac73a4cc7298 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:30.541057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1202/2373] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:03:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:03:31.179404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3894e44-ddc1-4958-adb6-20c86ddf239b None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:31.180453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3894e44-ddc1-4958-adb6-20c86ddf239b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:31.181092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3894e44-ddc1-4958-adb6-20c86ddf239b None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:31.188746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1172/2374] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:03:31 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:03:31.201245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1895bb-1d5c-4ada-97db-75c676fd43d9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:31.201245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1895bb-1d5c-4ada-97db-75c676fd43d9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:31.201245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1895bb-1d5c-4ada-97db-75c676fd43d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:31.234913 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7d1895bb-1d5c-4ada-97db-75c676fd43d9 None None] MFA Rules not processed for user `e8202ad6007246078c4d25365dc5a0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:31.236274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7d1895bb-1d5c-4ada-97db-75c676fd43d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:31.442869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1203/2375] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:03:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:31.497076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3de09-3aa9-4bcb-a382-1e6a63145a62 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:31.497654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3de09-3aa9-4bcb-a382-1e6a63145a62 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:31.497857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d3de09-3aa9-4bcb-a382-1e6a63145a62 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:31.499583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1173/2376] 10.222.0.25 () {54 vars in 961 bytes} [Tue Sep 13 07:03:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:03:31.505988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378d51-a4e8-4ee0-88a4-8f4e2fb03004 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:31.506569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378d51-a4e8-4ee0-88a4-8f4e2fb03004 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:31.507049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96378d51-a4e8-4ee0-88a4-8f4e2fb03004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:31.544729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-96378d51-a4e8-4ee0-88a4-8f4e2fb03004 None None] MFA Rules not processed for user `e8202ad6007246078c4d25365dc5a0f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:31.550857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-96378d51-a4e8-4ee0-88a4-8f4e2fb03004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:31.572206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1204/2377] 10.222.0.25 () {58 vars in 1058 bytes} [Tue Sep 13 07:03:31 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:31.581573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3de09-3aa9-4bcb-a382-1e6a63145a62 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:31.614171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d3de09-3aa9-4bcb-a382-1e6a63145a62 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:31.628043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edcb69-0d53-473f-b8af-c54e01e7efc1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:31.629313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edcb69-0d53-473f-b8af-c54e01e7efc1 service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:31.630456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edcb69-0d53-473f-b8af-c54e01e7efc1 service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:31.633218 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88edcb69-0d53-473f-b8af-c54e01e7efc1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:31.655509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edcb69-0d53-473f-b8af-c54e01e7efc1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:31.660633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88edcb69-0d53-473f-b8af-c54e01e7efc1 service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:31.686098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1174/2378] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:32.412051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96378d51-a4e8-4ee0-88a4-8f4e2fb03004 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:32.474707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:32.520897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96378d51-a4e8-4ee0-88a4-8f4e2fb03004 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:32.526274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:32.527345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:32.528693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:32.529339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] PATH_INFO: `/v3/users/51e461eb2801482d829cdc66eeb61f1c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:32.530728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] RBAC: Authorizing `identity:delete_user(user_id=51e461eb2801482d829cdc66eeb61f1c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:32.533161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:32.602384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51e461eb2801482d829cdc66eeb61f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:32.603348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51e461eb2801482d829cdc66eeb61f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:32.618068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51e461eb2801482d829cdc66eeb61f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:32.641025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51e461eb2801482d829cdc66eeb61f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:32.642387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51e461eb2801482d829cdc66eeb61f1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:32.666829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1205/2379] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:03:32 2022] DELETE /identity/v3/users/51e461eb2801482d829cdc66eeb61f1c => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:33.062507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edcb69-0d53-473f-b8af-c54e01e7efc1 service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:33.075412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88edcb69-0d53-473f-b8af-c54e01e7efc1 service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:33.222398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:33.223073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:33.223746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:33.224185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] PATH_INFO: `/v3/projects/f061702879f74ee1a6d1bb7722370cbc` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:33.226138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f061702879f74ee1a6d1bb7722370cbc)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:33.228419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:33.360741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f061702879f74ee1a6d1bb7722370cbc', 'request_id': 'req-cb5a1b7a-816a-4be5-af53-95658b500643'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.370964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f061702879f74ee1a6d1bb7722370cbc', 'request_id': 'req-cb5a1b7a-816a-4be5-af53-95658b500643'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.381617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f061702879f74ee1a6d1bb7722370cbc', 'request_id': 'req-cb5a1b7a-816a-4be5-af53-95658b500643'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.400469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:33.406384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] The token cache is being invalidate because project f061702879f74ee1a6d1bb7722370cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:03:33.407055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.407955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.413508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1175/2380] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:03:33 2022] DELETE /identity/v3/projects/f061702879f74ee1a6d1bb7722370cbc => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:33.414987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:33.416541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:33.449345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb20bbb-27f5-4583-a10f-13672005140d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:33.537919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:33.539880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:33.541129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:33.541778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] PATH_INFO: `/v3/users/85b5ab4fa4534e00b5deef113ac48523` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:33.544535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b5ab4fa4534e00b5deef113ac48523)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:33.548692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:33.652982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b5ab4fa4534e00b5deef113ac48523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.653858 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b5ab4fa4534e00b5deef113ac48523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.657122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:33.671079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b5ab4fa4534e00b5deef113ac48523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.709967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b5ab4fa4534e00b5deef113ac48523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.710758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b5ab4fa4534e00b5deef113ac48523'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:33.728401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1206/2381] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:03:33 2022] DELETE /identity/v3/users/85b5ab4fa4534e00b5deef113ac48523 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:33.929697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:33.932294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:33.954604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a1b7a-816a-4be5-af53-95658b500643 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:34.099750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:34.100780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:34.100780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:34.100989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] PATH_INFO: `/v3/users/5bd897d855c64ae28ec3f10f69e9e8b2` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:34.102104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bd897d855c64ae28ec3f10f69e9e8b2)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:34.106021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:34.124552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:34.126539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:34.176809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef660c-b1a1-418c-81cc-973769cbb595 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:34.199395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:34.201209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:34.202941 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:34.204329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] PATH_INFO: `/v3/projects/fdb4f665127d484192cac83dd1069cc7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:34.209554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdb4f665127d484192cac83dd1069cc7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:34.211685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:34.232776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bd897d855c64ae28ec3f10f69e9e8b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.250522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bd897d855c64ae28ec3f10f69e9e8b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.283260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdb4f665127d484192cac83dd1069cc7', 'request_id': 'req-37667800-b595-4020-8437-f5a39d5b2053'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.285459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bd897d855c64ae28ec3f10f69e9e8b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.288302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bd897d855c64ae28ec3f10f69e9e8b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.292882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdb4f665127d484192cac83dd1069cc7', 'request_id': 'req-37667800-b595-4020-8437-f5a39d5b2053'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.307943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdb4f665127d484192cac83dd1069cc7', 'request_id': 'req-37667800-b595-4020-8437-f5a39d5b2053'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.312314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1007e5b9-2f6d-4eb0-ac86-28fad393bc91 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bd897d855c64ae28ec3f10f69e9e8b2'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.319508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1176/2382] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:03:33 2022] DELETE /identity/v3/users/5bd897d855c64ae28ec3f10f69e9e8b2 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:34.335228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] The token cache is being invalidate because project fdb4f665127d484192cac83dd1069cc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:03:34.335608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.336347 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-37667800-b595-4020-8437-f5a39d5b2053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.337909 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1207/2383] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:03:34 2022] DELETE /identity/v3/projects/fdb4f665127d484192cac83dd1069cc7 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:34.343683 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0979c-6044-4701-81cf-fe4423c574f8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:34.344201 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0979c-6044-4701-81cf-fe4423c574f8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:34.344468 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0979c-6044-4701-81cf-fe4423c574f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:34.385692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9db0979c-6044-4701-81cf-fe4423c574f8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:34.387409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9db0979c-6044-4701-81cf-fe4423c574f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:34.493971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1177/2384] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:34.502306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98e07f-c6cd-4b51-8b64-90d9f9fa405e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:34.502573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98e07f-c6cd-4b51-8b64-90d9f9fa405e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:34.502863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da98e07f-c6cd-4b51-8b64-90d9f9fa405e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:34.552094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-da98e07f-c6cd-4b51-8b64-90d9f9fa405e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:34.553334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-da98e07f-c6cd-4b51-8b64-90d9f9fa405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:34.579145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1208/2385] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:34.585551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0979c-6044-4701-81cf-fe4423c574f8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:34.602933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0979c-6044-4701-81cf-fe4423c574f8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:34.608868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896cc2a-e535-4f1a-bcbc-2dc3ddac55bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:34.609721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896cc2a-e535-4f1a-bcbc-2dc3ddac55bd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:34.610066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896cc2a-e535-4f1a-bcbc-2dc3ddac55bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:34.610449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3896cc2a-e535-4f1a-bcbc-2dc3ddac55bd admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:34.611170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896cc2a-e535-4f1a-bcbc-2dc3ddac55bd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:03:34.611558 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896cc2a-e535-4f1a-bcbc-2dc3ddac55bd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:34.613618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3896cc2a-e535-4f1a-bcbc-2dc3ddac55bd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:34.621400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98e07f-c6cd-4b51-8b64-90d9f9fa405e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:34.624158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-da98e07f-c6cd-4b51-8b64-90d9f9fa405e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:34.628252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1178/2386] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:03:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:03:34.638608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896cc2a-e535-4f1a-bcbc-2dc3ddac55bd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:34.647069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da98e07f-c6cd-4b51-8b64-90d9f9fa405e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:34.654210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3896cc2a-e535-4f1a-bcbc-2dc3ddac55bd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:34.658735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:34.659796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:34.660240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:34.660725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] PATH_INFO: `/v3/projects/81082b2f3a984d09baac046de385ea96` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:34.662862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=81082b2f3a984d09baac046de385ea96)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:34.665280 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:34.670962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba489e8c-35a5-4160-8791-9393cf36cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:34.671422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba489e8c-35a5-4160-8791-9393cf36cc81 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:34.671816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba489e8c-35a5-4160-8791-9393cf36cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:34.671971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba489e8c-35a5-4160-8791-9393cf36cc81 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:34.672616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba489e8c-35a5-4160-8791-9393cf36cc81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:34.674586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba489e8c-35a5-4160-8791-9393cf36cc81 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:34.691294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:34.735777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1179/2387] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:03:34 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:34.739428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba489e8c-35a5-4160-8791-9393cf36cc81 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:34.770838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '81082b2f3a984d09baac046de385ea96', 'request_id': 'req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.783306 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '81082b2f3a984d09baac046de385ea96', 'request_id': 'req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.802896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '81082b2f3a984d09baac046de385ea96', 'request_id': 'req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.830960 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba489e8c-35a5-4160-8791-9393cf36cc81 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:34.834451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] The token cache is being invalidate because project 81082b2f3a984d09baac046de385ea96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:03:34.834751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.837555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5d8d707b-90a1-4cc1-8d12-f4a7bd083174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:34.839689 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1209/2388] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:03:34 2022] DELETE /identity/v3/projects/81082b2f3a984d09baac046de385ea96 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:03:34.844535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a87ac3-9965-47d5-8e19-6fd8daec6b80 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:34.844827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a87ac3-9965-47d5-8e19-6fd8daec6b80 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:34.845111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a87ac3-9965-47d5-8e19-6fd8daec6b80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:34.881146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-83a87ac3-9965-47d5-8e19-6fd8daec6b80 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:34.883027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-83a87ac3-9965-47d5-8e19-6fd8daec6b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:34.920190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=3,keystone:DELETE=17,keystone:SELECT=80 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:34.997026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1210/2389] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:35.006123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d2db0-dc34-44dc-9ac0-9dcf2b3e6408 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.006473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d2db0-dc34-44dc-9ac0-9dcf2b3e6408 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.006769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d2db0-dc34-44dc-9ac0-9dcf2b3e6408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.041942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ca5d2db0-dc34-44dc-9ac0-9dcf2b3e6408 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:35.043343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ca5d2db0-dc34-44dc-9ac0-9dcf2b3e6408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:35.057011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1211/2390] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:03:35.058552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d2db0-dc34-44dc-9ac0-9dcf2b3e6408 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.084725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d2db0-dc34-44dc-9ac0-9dcf2b3e6408 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.091667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6210cd-1449-406d-9fc6-c19a9e0c1eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.092318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6210cd-1449-406d-9fc6-c19a9e0c1eff admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.092602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6210cd-1449-406d-9fc6-c19a9e0c1eff admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.092879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6210cd-1449-406d-9fc6-c19a9e0c1eff admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.093547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6210cd-1449-406d-9fc6-c19a9e0c1eff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:03:35.093855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6210cd-1449-406d-9fc6-c19a9e0c1eff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.095536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6210cd-1449-406d-9fc6-c19a9e0c1eff admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.107089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521551f-bfd2-4bcc-9508-0bc3e7a1eca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.107901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521551f-bfd2-4bcc-9508-0bc3e7a1eca3 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.108185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521551f-bfd2-4bcc-9508-0bc3e7a1eca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.108531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f521551f-bfd2-4bcc-9508-0bc3e7a1eca3 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.109263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521551f-bfd2-4bcc-9508-0bc3e7a1eca3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.116674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f521551f-bfd2-4bcc-9508-0bc3e7a1eca3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.130201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1212/2391] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:03:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:03:35.134567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6210cd-1449-406d-9fc6-c19a9e0c1eff admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.149698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6210cd-1449-406d-9fc6-c19a9e0c1eff admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.155890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06644f-3e3f-4c43-95f3-05ad28d889c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.157225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06644f-3e3f-4c43-95f3-05ad28d889c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.157773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06644f-3e3f-4c43-95f3-05ad28d889c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.158177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac06644f-3e3f-4c43-95f3-05ad28d889c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.158839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06644f-3e3f-4c43-95f3-05ad28d889c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.160833 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac06644f-3e3f-4c43-95f3-05ad28d889c2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.163397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1180/2392] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:03:34 2022] POST /identity/v3/users => generated 287 bytes in 425 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:35.167777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521551f-bfd2-4bcc-9508-0bc3e7a1eca3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.169467 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f521551f-bfd2-4bcc-9508-0bc3e7a1eca3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:35.183174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f521551f-bfd2-4bcc-9508-0bc3e7a1eca3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.189599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed44a5-e4c5-484d-b0e0-7432b1ce297f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.190266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed44a5-e4c5-484d-b0e0-7432b1ce297f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.190557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed44a5-e4c5-484d-b0e0-7432b1ce297f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.190857 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ed44a5-e4c5-484d-b0e0-7432b1ce297f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.191484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed44a5-e4c5-484d-b0e0-7432b1ce297f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.193108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ed44a5-e4c5-484d-b0e0-7432b1ce297f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.206330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1181/2393] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:35.210160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed44a5-e4c5-484d-b0e0-7432b1ce297f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.216038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1213/2394] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:03:35 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:35.220015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06644f-3e3f-4c43-95f3-05ad28d889c2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.232592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ed44a5-e4c5-484d-b0e0-7432b1ce297f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.326683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac06644f-3e3f-4c43-95f3-05ad28d889c2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.328693 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55812105-3eec-4a02-b34d-a84d6b29f042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.329525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55812105-3eec-4a02-b34d-a84d6b29f042 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.329971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55812105-3eec-4a02-b34d-a84d6b29f042 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.330446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55812105-3eec-4a02-b34d-a84d6b29f042 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.331104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55812105-3eec-4a02-b34d-a84d6b29f042 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.332713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55812105-3eec-4a02-b34d-a84d6b29f042 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.335210 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a795c-59f8-4bda-9756-aa1ef4d93c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.335210 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a795c-59f8-4bda-9756-aa1ef4d93c76 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.335210 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a795c-59f8-4bda-9756-aa1ef4d93c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.335210 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a795c-59f8-4bda-9756-aa1ef4d93c76 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.335632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a795c-59f8-4bda-9756-aa1ef4d93c76 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.338617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460a795c-59f8-4bda-9756-aa1ef4d93c76 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.348795 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1182/2395] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:03:35.352813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55812105-3eec-4a02-b34d-a84d6b29f042 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.364600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55812105-3eec-4a02-b34d-a84d6b29f042 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.370222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265b35b-eb36-42fb-aa11-ddd6f0f0d7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.371190 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265b35b-eb36-42fb-aa11-ddd6f0f0d7da admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.371549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265b35b-eb36-42fb-aa11-ddd6f0f0d7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.371890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265b35b-eb36-42fb-aa11-ddd6f0f0d7da admin admin] PATH_INFO: `/v3/projects/09236b40298e4a37b65d89ae8943ec97/users/c794459f02244cadab531e5bf3900833/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.375628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:35.406801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1214/2396] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:03:35 2022] POST /identity/v3/users => generated 287 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:35.420772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a795c-59f8-4bda-9756-aa1ef4d93c76 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.423185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265b35b-eb36-42fb-aa11-ddd6f0f0d7da admin admin] RBAC: Authorizing `identity:create_grant(project_id=09236b40298e4a37b65d89ae8943ec97, user_id=c794459f02244cadab531e5bf3900833, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.426060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265b35b-eb36-42fb-aa11-ddd6f0f0d7da admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.435944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a795c-59f8-4bda-9756-aa1ef4d93c76 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.462487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1183/2397] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:03:35 2022] PUT /identity/v3/projects/09236b40298e4a37b65d89ae8943ec97/users/c794459f02244cadab531e5bf3900833/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:35.471644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629bfa8-a0ec-4a83-97a5-faefd5d10255 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.472265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629bfa8-a0ec-4a83-97a5-faefd5d10255 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.472547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f629bfa8-a0ec-4a83-97a5-faefd5d10255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.511685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f629bfa8-a0ec-4a83-97a5-faefd5d10255 None None] MFA Rules not processed for user `c794459f02244cadab531e5bf3900833`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:35.513217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f629bfa8-a0ec-4a83-97a5-faefd5d10255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:35.514265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e383-1227-46d0-bb36-d3f95fc47721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.515027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e383-1227-46d0-bb36-d3f95fc47721 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.530139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e383-1227-46d0-bb36-d3f95fc47721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.530715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8202e383-1227-46d0-bb36-d3f95fc47721 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.531616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202e383-1227-46d0-bb36-d3f95fc47721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.533274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8202e383-1227-46d0-bb36-d3f95fc47721 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.552481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1215/2398] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:35.556987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e383-1227-46d0-bb36-d3f95fc47721 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.568300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8202e383-1227-46d0-bb36-d3f95fc47721 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.575562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82156d17-9b6a-4001-ae4b-1b94c0895f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.576357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82156d17-9b6a-4001-ae4b-1b94c0895f35 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.576704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82156d17-9b6a-4001-ae4b-1b94c0895f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.576982 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82156d17-9b6a-4001-ae4b-1b94c0895f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.577633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82156d17-9b6a-4001-ae4b-1b94c0895f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.579419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82156d17-9b6a-4001-ae4b-1b94c0895f35 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.581905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:35.592402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1216/2399] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:35 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:35.595654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1184/2400] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:03:35.598293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82156d17-9b6a-4001-ae4b-1b94c0895f35 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.610971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629bfa8-a0ec-4a83-97a5-faefd5d10255 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.610971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f629bfa8-a0ec-4a83-97a5-faefd5d10255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:35.611298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82156d17-9b6a-4001-ae4b-1b94c0895f35 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.614980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b38097-fa01-4f7d-a6d4-8343087ae411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.615649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b38097-fa01-4f7d-a6d4-8343087ae411 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.615926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b38097-fa01-4f7d-a6d4-8343087ae411 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.616198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b38097-fa01-4f7d-a6d4-8343087ae411 admin admin] PATH_INFO: `/v3/projects/5006f7a1770f4d9a96cbec0715987237/users/7e0774e048684f6b8f79ee44f1c4ff7c/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.640782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b38097-fa01-4f7d-a6d4-8343087ae411 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5006f7a1770f4d9a96cbec0715987237, user_id=7e0774e048684f6b8f79ee44f1c4ff7c, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.644173 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b38097-fa01-4f7d-a6d4-8343087ae411 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.660191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1217/2401] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:03:35 2022] PUT /identity/v3/projects/5006f7a1770f4d9a96cbec0715987237/users/7e0774e048684f6b8f79ee44f1c4ff7c/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:35.671174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9937680-8b88-4e90-991f-c844b4064894 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.671732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9937680-8b88-4e90-991f-c844b4064894 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.672013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9937680-8b88-4e90-991f-c844b4064894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.695094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f629bfa8-a0ec-4a83-97a5-faefd5d10255 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.722559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f9937680-8b88-4e90-991f-c844b4064894 None None] MFA Rules not processed for user `7e0774e048684f6b8f79ee44f1c4ff7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:35.725871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f9937680-8b88-4e90-991f-c844b4064894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:35.774345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ba1a8f-701d-4cd8-9727-ff18e462f1c9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.774345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba1a8f-701d-4cd8-9727-ff18e462f1c9 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.774345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba1a8f-701d-4cd8-9727-ff18e462f1c9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.774345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ba1a8f-701d-4cd8-9727-ff18e462f1c9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.820760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:35.842105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1218/2402] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:03:35.858854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9937680-8b88-4e90-991f-c844b4064894 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:35.861000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f9937680-8b88-4e90-991f-c844b4064894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:35.865565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba1a8f-701d-4cd8-9727-ff18e462f1c9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.867604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ba1a8f-701d-4cd8-9727-ff18e462f1c9 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.877413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9937680-8b88-4e90-991f-c844b4064894 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:35.879126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1185/2403] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:35.882073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01ffe7-6cf3-464b-b57a-0e90ebc62581 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:35.882796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01ffe7-6cf3-464b-b57a-0e90ebc62581 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:35.883075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01ffe7-6cf3-464b-b57a-0e90ebc62581 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:35.883450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af01ffe7-6cf3-464b-b57a-0e90ebc62581 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:35.893529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01ffe7-6cf3-464b-b57a-0e90ebc62581 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:35.895154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af01ffe7-6cf3-464b-b57a-0e90ebc62581 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:35.926370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1219/2404] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:03:37.095949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af25c3-3284-4aaa-b0d9-f702df22387b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:37.097082 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af25c3-3284-4aaa-b0d9-f702df22387b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:37.097675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8af25c3-3284-4aaa-b0d9-f702df22387b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:37.121367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d8af25c3-3284-4aaa-b0d9-f702df22387b None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:37.122649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d8af25c3-3284-4aaa-b0d9-f702df22387b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:37.208863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1186/2405] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:03:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:37.221401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01ffe7-6cf3-464b-b57a-0e90ebc62581 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:37.222980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-af01ffe7-6cf3-464b-b57a-0e90ebc62581 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:37.298308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af01ffe7-6cf3-464b-b57a-0e90ebc62581 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:37.303833 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb61214-0479-4884-982a-877c2d96abec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:37.304560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb61214-0479-4884-982a-877c2d96abec service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:37.304773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb61214-0479-4884-982a-877c2d96abec service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:37.305044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb61214-0479-4884-982a-877c2d96abec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:37.315852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb61214-0479-4884-982a-877c2d96abec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:37.317569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb61214-0479-4884-982a-877c2d96abec service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:37.329902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1220/2406] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:03:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:03:37.383366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159f2d1-35ed-4559-b0ef-93cd4c30721a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:37.384100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159f2d1-35ed-4559-b0ef-93cd4c30721a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:37.384277 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a159f2d1-35ed-4559-b0ef-93cd4c30721a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:37.420670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a159f2d1-35ed-4559-b0ef-93cd4c30721a None None] MFA Rules not processed for user `7e0774e048684f6b8f79ee44f1c4ff7c`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:37.421991 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a159f2d1-35ed-4559-b0ef-93cd4c30721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:37.441798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1187/2407] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:37 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:39.610323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb61214-0479-4884-982a-877c2d96abec service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:39.742030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:39.809660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb61214-0479-4884-982a-877c2d96abec service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:39.820399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:39.821479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:39.822506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:39.823492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] PATH_INFO: `/v3/users/c794459f02244cadab531e5bf3900833` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:39.825761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c794459f02244cadab531e5bf3900833)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:39.829740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:39.954732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c794459f02244cadab531e5bf3900833'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:39.956747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c794459f02244cadab531e5bf3900833'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:39.971687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c794459f02244cadab531e5bf3900833'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:39.996383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c794459f02244cadab531e5bf3900833'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:39.997784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c794459f02244cadab531e5bf3900833'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:40.019625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1221/2408] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:03:39 2022] DELETE /identity/v3/users/c794459f02244cadab531e5bf3900833 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:40.024414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614fcae-5965-4555-bb91-7391d8994b7f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:40.025018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614fcae-5965-4555-bb91-7391d8994b7f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:40.025018 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9614fcae-5965-4555-bb91-7391d8994b7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:40.068107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9614fcae-5965-4555-bb91-7391d8994b7f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:40.069863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9614fcae-5965-4555-bb91-7391d8994b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:40.160829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1188/2409] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:40.166056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:40.168609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:40.277974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29886160-00ca-41ba-9c10-2ac51b61a72c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:40.287417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b083c6-8228-4339-832e-44e8816f7731 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:40.288076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b083c6-8228-4339-832e-44e8816f7731 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:40.288341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b083c6-8228-4339-832e-44e8816f7731 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:40.288613 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b083c6-8228-4339-832e-44e8816f7731 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:40.310000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b083c6-8228-4339-832e-44e8816f7731 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:40.311370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b083c6-8228-4339-832e-44e8816f7731 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:40.327292 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1222/2410] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:41.881168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614fcae-5965-4555-bb91-7391d8994b7f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:41.883501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9614fcae-5965-4555-bb91-7391d8994b7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:41.901437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9614fcae-5965-4555-bb91-7391d8994b7f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:41.908005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:41.908962 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:41.909542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:41.913798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] PATH_INFO: `/v3/projects/09236b40298e4a37b65d89ae8943ec97` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:41.915670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] RBAC: Authorizing `identity:delete_project(project_id=09236b40298e4a37b65d89ae8943ec97)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:41.918156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:42.044413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09236b40298e4a37b65d89ae8943ec97', 'request_id': 'req-2b942f52-65b4-4cc5-94f0-c463599a067a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.054133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09236b40298e4a37b65d89ae8943ec97', 'request_id': 'req-2b942f52-65b4-4cc5-94f0-c463599a067a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.064432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09236b40298e4a37b65d89ae8943ec97', 'request_id': 'req-2b942f52-65b4-4cc5-94f0-c463599a067a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.096796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] The token cache is being invalidate because project 09236b40298e4a37b65d89ae8943ec97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:03:42.097187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.097867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.100557 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1189/2411] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:03:41 2022] DELETE /identity/v3/projects/09236b40298e4a37b65d89ae8943ec97 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:42.105893 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f5db4-82be-4641-a3c0-4d12013e798b None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:42.107513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f5db4-82be-4641-a3c0-4d12013e798b None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:42.107904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385f5db4-82be-4641-a3c0-4d12013e798b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:42.149314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-385f5db4-82be-4641-a3c0-4d12013e798b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:42.151505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-385f5db4-82be-4641-a3c0-4d12013e798b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:42.200547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:42.202162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:42.220478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:42.337688 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b942f52-65b4-4cc5-94f0-c463599a067a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:42.346120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1223/2412] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:42.351389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:42.351389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:42.351389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:42.351389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] PATH_INFO: `/v3/users/b7a67839b2cb41a38eef7e288dee6f12` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:42.351822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7a67839b2cb41a38eef7e288dee6f12)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:42.352276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:42.353600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28536d-b95a-46fd-a01d-1f225c54f1ab None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:42.354611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28536d-b95a-46fd-a01d-1f225c54f1ab None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:42.355169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db28536d-b95a-46fd-a01d-1f225c54f1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:42.365088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:42.389904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-db28536d-b95a-46fd-a01d-1f225c54f1ab None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:42.392513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-db28536d-b95a-46fd-a01d-1f225c54f1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:42.422509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1224/2413] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:03:42.426757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28536d-b95a-46fd-a01d-1f225c54f1ab None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:42.438270 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7a67839b2cb41a38eef7e288dee6f12'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.445641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db28536d-b95a-46fd-a01d-1f225c54f1ab None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:42.461605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22028136-86ac-4a46-9e53-b323d7557fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:42.462590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22028136-86ac-4a46-9e53-b323d7557fc4 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:42.463062 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22028136-86ac-4a46-9e53-b323d7557fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:42.463515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7a67839b2cb41a38eef7e288dee6f12'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.463649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22028136-86ac-4a46-9e53-b323d7557fc4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:42.465573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22028136-86ac-4a46-9e53-b323d7557fc4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:03:42.466447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22028136-86ac-4a46-9e53-b323d7557fc4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:42.468879 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22028136-86ac-4a46-9e53-b323d7557fc4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:42.483854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7a67839b2cb41a38eef7e288dee6f12'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.485718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7a67839b2cb41a38eef7e288dee6f12'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.496527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1225/2414] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:03:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:03:42.497501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22028136-86ac-4a46-9e53-b323d7557fc4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:42.499033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1cff9827-5d75-4065-9982-a919f4cafd59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7a67839b2cb41a38eef7e288dee6f12'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:42.509143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1190/2415] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:03:42 2022] DELETE /identity/v3/users/b7a67839b2cb41a38eef7e288dee6f12 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:42.509348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22028136-86ac-4a46-9e53-b323d7557fc4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:42.511693 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62203d84-34fc-4b14-8b8a-93b5207a51a4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:42.511989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62203d84-34fc-4b14-8b8a-93b5207a51a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:42.512263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62203d84-34fc-4b14-8b8a-93b5207a51a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:42.570381 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-62203d84-34fc-4b14-8b8a-93b5207a51a4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:42.574961 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-62203d84-34fc-4b14-8b8a-93b5207a51a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:42.646417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f4e3e-79cd-4567-91d1-00a57cfc6276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:42.647236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f4e3e-79cd-4567-91d1-00a57cfc6276 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:42.648278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f4e3e-79cd-4567-91d1-00a57cfc6276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:42.648278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6f4e3e-79cd-4567-91d1-00a57cfc6276 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:42.648787 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f4e3e-79cd-4567-91d1-00a57cfc6276 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:42.651440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6f4e3e-79cd-4567-91d1-00a57cfc6276 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:42.672720 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1191/2416] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:42.684888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62203d84-34fc-4b14-8b8a-93b5207a51a4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:42.687435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-62203d84-34fc-4b14-8b8a-93b5207a51a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:42.719779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1226/2417] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:03:42 2022] POST /identity/v3/projects => generated 369 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:42.724686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f4e3e-79cd-4567-91d1-00a57cfc6276 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:42.726981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0f6f4e3e-79cd-4567-91d1-00a57cfc6276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:42.751002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:42.838043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62203d84-34fc-4b14-8b8a-93b5207a51a4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:42.874619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6f4e3e-79cd-4567-91d1-00a57cfc6276 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:42.891326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b9287-39cd-441c-b917-3c7cfdd13899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:42.892716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b9287-39cd-441c-b917-3c7cfdd13899 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:42.893225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b9287-39cd-441c-b917-3c7cfdd13899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:42.893696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242b9287-39cd-441c-b917-3c7cfdd13899 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:42.895032 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b9287-39cd-441c-b917-3c7cfdd13899 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:42.899349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242b9287-39cd-441c-b917-3c7cfdd13899 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:42.947975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1227/2418] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:03:42 2022] POST /identity/v3/users => generated 285 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:42.950626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d271a0-13ca-4e37-9f44-b5a5e6a9a69e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:42.951928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d271a0-13ca-4e37-9f44-b5a5e6a9a69e service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:42.952377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d271a0-13ca-4e37-9f44-b5a5e6a9a69e service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:42.952893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d271a0-13ca-4e37-9f44-b5a5e6a9a69e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:42.953373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b9287-39cd-441c-b917-3c7cfdd13899 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:42.966257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242b9287-39cd-441c-b917-3c7cfdd13899 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:42.967485 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d271a0-13ca-4e37-9f44-b5a5e6a9a69e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:42.969187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d271a0-13ca-4e37-9f44-b5a5e6a9a69e service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:42.972072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df66380-4503-4a17-8472-d386f3348768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:42.972932 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df66380-4503-4a17-8472-d386f3348768 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:42.973338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df66380-4503-4a17-8472-d386f3348768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:42.973704 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df66380-4503-4a17-8472-d386f3348768 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:42.975159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df66380-4503-4a17-8472-d386f3348768 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:42.977073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df66380-4503-4a17-8472-d386f3348768 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:42.988105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1192/2419] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:03:42.990539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1228/2420] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:42.993567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df66380-4503-4a17-8472-d386f3348768 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:43.011096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df66380-4503-4a17-8472-d386f3348768 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:43.017368 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c66e7a6-4f1b-4dab-9160-f00b26a38c78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:43.018524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66e7a6-4f1b-4dab-9160-f00b26a38c78 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:43.018811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66e7a6-4f1b-4dab-9160-f00b26a38c78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:43.019278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c66e7a6-4f1b-4dab-9160-f00b26a38c78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:43.019942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66e7a6-4f1b-4dab-9160-f00b26a38c78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:43.034441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c66e7a6-4f1b-4dab-9160-f00b26a38c78 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:43.050942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1229/2421] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:42 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:03:43.054860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d271a0-13ca-4e37-9f44-b5a5e6a9a69e service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:43.069403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d271a0-13ca-4e37-9f44-b5a5e6a9a69e service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:43.081678 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51145fc2-b907-480b-b5bc-93aab778bf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:43.082340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145fc2-b907-480b-b5bc-93aab778bf9a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:43.082627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145fc2-b907-480b-b5bc-93aab778bf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:43.082914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51145fc2-b907-480b-b5bc-93aab778bf9a admin admin] PATH_INFO: `/v3/projects/21a4c6cf46844a269832e0b6f919ebf9/users/3684e8422d6b4c53836519016dc0e8da/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:43.119275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51145fc2-b907-480b-b5bc-93aab778bf9a admin admin] RBAC: Authorizing `identity:create_grant(project_id=21a4c6cf46844a269832e0b6f919ebf9, user_id=3684e8422d6b4c53836519016dc0e8da, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:43.122256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51145fc2-b907-480b-b5bc-93aab778bf9a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:43.161103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1193/2422] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:03:43 2022] PUT /identity/v3/projects/21a4c6cf46844a269832e0b6f919ebf9/users/3684e8422d6b4c53836519016dc0e8da/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:43.179081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7d0b4-5e0a-4482-b411-fae0fc8230eb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:43.179081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7d0b4-5e0a-4482-b411-fae0fc8230eb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:43.179081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd7d0b4-5e0a-4482-b411-fae0fc8230eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:43.240095 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8fd7d0b4-5e0a-4482-b411-fae0fc8230eb None None] MFA Rules not processed for user `3684e8422d6b4c53836519016dc0e8da`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:43.241525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8fd7d0b4-5e0a-4482-b411-fae0fc8230eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:43.287044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:43.416658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1230/2423] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:43.426482 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51145fc2-b907-480b-b5bc-93aab778bf9a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:43.438692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-51145fc2-b907-480b-b5bc-93aab778bf9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:43.473390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7d0b4-5e0a-4482-b411-fae0fc8230eb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:43.474934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8fd7d0b4-5e0a-4482-b411-fae0fc8230eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:43.612894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd7d0b4-5e0a-4482-b411-fae0fc8230eb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:43.619902 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51145fc2-b907-480b-b5bc-93aab778bf9a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:43.622337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:43.623338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:43.623812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:43.624240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] PATH_INFO: `/v3/projects/57c5f928bfc8427cbfc7dd20e7a773ba` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:43.625545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57c5f928bfc8427cbfc7dd20e7a773ba)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:43.627296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db40eda9-d01a-4b5b-8d8f-36fa3f42c308 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3505c8965c4a42e493a2f5916cab56dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:43.627928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40eda9-d01a-4b5b-8d8f-36fa3f42c308 service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:43.628206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40eda9-d01a-4b5b-8d8f-36fa3f42c308 service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:43.628410 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:43.628523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db40eda9-d01a-4b5b-8d8f-36fa3f42c308 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:43.654986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40eda9-d01a-4b5b-8d8f-36fa3f42c308 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:43.656503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db40eda9-d01a-4b5b-8d8f-36fa3f42c308 service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:43.669403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1194/2424] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:43 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:43.719295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57c5f928bfc8427cbfc7dd20e7a773ba', 'request_id': 'req-2c90b0fa-5f5e-4b57-ad52-181658637ba2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:43.728050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57c5f928bfc8427cbfc7dd20e7a773ba', 'request_id': 'req-2c90b0fa-5f5e-4b57-ad52-181658637ba2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:43.735116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57c5f928bfc8427cbfc7dd20e7a773ba', 'request_id': 'req-2c90b0fa-5f5e-4b57-ad52-181658637ba2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:43.757172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] The token cache is being invalidate because project 57c5f928bfc8427cbfc7dd20e7a773ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:03:43.757172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:43.757172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2c90b0fa-5f5e-4b57-ad52-181658637ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:03:43.758658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1231/2425] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:03:43 2022] DELETE /identity/v3/projects/57c5f928bfc8427cbfc7dd20e7a773ba => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:43.764083 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f58ca-dbc7-4b9f-8679-c387b0bfac34 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:43.764356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f58ca-dbc7-4b9f-8679-c387b0bfac34 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:43.764644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f58ca-dbc7-4b9f-8679-c387b0bfac34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:43.800148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-603f58ca-dbc7-4b9f-8679-c387b0bfac34 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:43.802060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-603f58ca-dbc7-4b9f-8679-c387b0bfac34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:43.903849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1195/2426] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:43.911460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c34e3-e2ed-4a0f-b22a-0376cd48819a None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:43.912358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c34e3-e2ed-4a0f-b22a-0376cd48819a None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:43.913020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b4c34e3-e2ed-4a0f-b22a-0376cd48819a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:43.951973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2b4c34e3-e2ed-4a0f-b22a-0376cd48819a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:43.953535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2b4c34e3-e2ed-4a0f-b22a-0376cd48819a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:43.973535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1232/2427] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:43.978202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f58ca-dbc7-4b9f-8679-c387b0bfac34 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:43.997700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:44.011437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603f58ca-dbc7-4b9f-8679-c387b0bfac34 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:44.030981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20ad06-47e3-4f75-b8e9-0bce8e0e510c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:44.033745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20ad06-47e3-4f75-b8e9-0bce8e0e510c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:44.036715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20ad06-47e3-4f75-b8e9-0bce8e0e510c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:44.037098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20ad06-47e3-4f75-b8e9-0bce8e0e510c admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:44.037932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20ad06-47e3-4f75-b8e9-0bce8e0e510c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:03:44.038360 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20ad06-47e3-4f75-b8e9-0bce8e0e510c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:44.040145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f20ad06-47e3-4f75-b8e9-0bce8e0e510c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:44.073396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1196/2428] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:03:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:03:44.077788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c34e3-e2ed-4a0f-b22a-0376cd48819a None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:44.089314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b4c34e3-e2ed-4a0f-b22a-0376cd48819a None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:44.096417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352c628-6b18-4bec-9dda-4c784e639286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:44.099589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352c628-6b18-4bec-9dda-4c784e639286 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:44.102172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352c628-6b18-4bec-9dda-4c784e639286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:44.102849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2352c628-6b18-4bec-9dda-4c784e639286 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:44.103775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352c628-6b18-4bec-9dda-4c784e639286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:44.106499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2352c628-6b18-4bec-9dda-4c784e639286 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:44.164651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1233/2429] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:03:44 2022] POST /identity/v3/projects => generated 373 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:44.168588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20ad06-47e3-4f75-b8e9-0bce8e0e510c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:44.183547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20ad06-47e3-4f75-b8e9-0bce8e0e510c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:44.283113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb6c7b-e0da-472f-90fe-c756c3d68958 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:44.283781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb6c7b-e0da-472f-90fe-c756c3d68958 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:44.284177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb6c7b-e0da-472f-90fe-c756c3d68958 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:44.284459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb6c7b-e0da-472f-90fe-c756c3d68958 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:44.285185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb6c7b-e0da-472f-90fe-c756c3d68958 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:44.288403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb6c7b-e0da-472f-90fe-c756c3d68958 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:44.355929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1197/2430] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:03:44 2022] POST /identity/v3/users => generated 287 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:03:44.357234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352c628-6b18-4bec-9dda-4c784e639286 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:44.379048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2352c628-6b18-4bec-9dda-4c784e639286 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:44.389649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceaffb-ae03-4bd0-a91b-e58c44cc18cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:44.390250 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceaffb-ae03-4bd0-a91b-e58c44cc18cc admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:44.390580 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceaffb-ae03-4bd0-a91b-e58c44cc18cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:44.390855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ceaffb-ae03-4bd0-a91b-e58c44cc18cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:44.394894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ceaffb-ae03-4bd0-a91b-e58c44cc18cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:44.396564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ceaffb-ae03-4bd0-a91b-e58c44cc18cc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:44.429829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1234/2431] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:44.431748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb6c7b-e0da-472f-90fe-c756c3d68958 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:44.446297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb6c7b-e0da-472f-90fe-c756c3d68958 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:44.457710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87feb10f-a88e-42ec-84d3-eb01daa4d40a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:44.458788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb10f-a88e-42ec-84d3-eb01daa4d40a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:44.459296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb10f-a88e-42ec-84d3-eb01daa4d40a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:44.460096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87feb10f-a88e-42ec-84d3-eb01daa4d40a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:44.461227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feb10f-a88e-42ec-84d3-eb01daa4d40a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:44.465161 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87feb10f-a88e-42ec-84d3-eb01daa4d40a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:44.487527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1198/2432] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:44 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:03:44.494273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceaffb-ae03-4bd0-a91b-e58c44cc18cc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:44.507663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ceaffb-ae03-4bd0-a91b-e58c44cc18cc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:44.521531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06244605-1fa3-4a5b-b6dd-a7b1a0fbb26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:44.522323 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06244605-1fa3-4a5b-b6dd-a7b1a0fbb26a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:44.522606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06244605-1fa3-4a5b-b6dd-a7b1a0fbb26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:44.522884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06244605-1fa3-4a5b-b6dd-a7b1a0fbb26a admin admin] PATH_INFO: `/v3/projects/8e08bdf8d28f4a5093df2c5b26ccb9e6/users/a04d90d3f1e04a00b2ace1893ce5ef71/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:44.539277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:44.548656 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06244605-1fa3-4a5b-b6dd-a7b1a0fbb26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e08bdf8d28f4a5093df2c5b26ccb9e6, user_id=a04d90d3f1e04a00b2ace1893ce5ef71, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:44.551448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06244605-1fa3-4a5b-b6dd-a7b1a0fbb26a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:44.601316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1235/2433] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:03:44 2022] PUT /identity/v3/projects/8e08bdf8d28f4a5093df2c5b26ccb9e6/users/a04d90d3f1e04a00b2ace1893ce5ef71/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:44.613175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5474019-0115-4d77-98a6-e44b6de51d91 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:44.613175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5474019-0115-4d77-98a6-e44b6de51d91 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:44.613664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5474019-0115-4d77-98a6-e44b6de51d91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:44.691282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e5474019-0115-4d77-98a6-e44b6de51d91 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:44.698542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e5474019-0115-4d77-98a6-e44b6de51d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:44.865222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1199/2434] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:44.875511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06244605-1fa3-4a5b-b6dd-a7b1a0fbb26a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:44.879655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-06244605-1fa3-4a5b-b6dd-a7b1a0fbb26a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:45.015519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06244605-1fa3-4a5b-b6dd-a7b1a0fbb26a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:45.029924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049c209-2126-4ef4-bcde-0f4b691628f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:45.030597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049c209-2126-4ef4-bcde-0f4b691628f0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:45.030878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049c209-2126-4ef4-bcde-0f4b691628f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:45.031154 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049c209-2126-4ef4-bcde-0f4b691628f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:45.043647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049c209-2126-4ef4-bcde-0f4b691628f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:45.045686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049c209-2126-4ef4-bcde-0f4b691628f0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:45.070278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1236/2435] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:03:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:03:53.375236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b393ded-3dd7-4056-b73d-88b49d892b32 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:53.376889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b393ded-3dd7-4056-b73d-88b49d892b32 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:53.377374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b393ded-3dd7-4056-b73d-88b49d892b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:53.433916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9b393ded-3dd7-4056-b73d-88b49d892b32 None None] MFA Rules not processed for user `a04d90d3f1e04a00b2ace1893ce5ef71`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:03:53.435436 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b393ded-3dd7-4056-b73d-88b49d892b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:03:53.573327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:03:53.580874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1200/2436] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:03:53 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:03:53.586295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049c209-2126-4ef4-bcde-0f4b691628f0 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:53.611698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049c209-2126-4ef4-bcde-0f4b691628f0 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:53.627189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb1b60-7ccc-48ad-9632-08a64fb5bed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:53.628791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb1b60-7ccc-48ad-9632-08a64fb5bed0 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:53.629472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb1b60-7ccc-48ad-9632-08a64fb5bed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:53.629967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb1b60-7ccc-48ad-9632-08a64fb5bed0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:53.630986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb1b60-7ccc-48ad-9632-08a64fb5bed0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:53.633198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb1b60-7ccc-48ad-9632-08a64fb5bed0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:53.681147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1237/2437] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:03:53 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:53.685042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b393ded-3dd7-4056-b73d-88b49d892b32 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:53.700237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b393ded-3dd7-4056-b73d-88b49d892b32 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:53.776016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e20afa-ee16-4c5b-a850-721ed8137de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:53.776693 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e20afa-ee16-4c5b-a850-721ed8137de9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:53.776895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e20afa-ee16-4c5b-a850-721ed8137de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:53.777171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e20afa-ee16-4c5b-a850-721ed8137de9 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:53.777816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e20afa-ee16-4c5b-a850-721ed8137de9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:53.779972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e20afa-ee16-4c5b-a850-721ed8137de9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:53.808157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1201/2438] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:03:53 2022] POST /identity/v3/users => generated 292 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:03:53.811762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb1b60-7ccc-48ad-9632-08a64fb5bed0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:53.823872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb1b60-7ccc-48ad-9632-08a64fb5bed0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:53.831005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d85ea5-95cc-4e7b-9007-d946a2132dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:53.831720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d85ea5-95cc-4e7b-9007-d946a2132dbf admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:53.832005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d85ea5-95cc-4e7b-9007-d946a2132dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:53.832299 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d85ea5-95cc-4e7b-9007-d946a2132dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:53.832944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d85ea5-95cc-4e7b-9007-d946a2132dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:53.834630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d85ea5-95cc-4e7b-9007-d946a2132dbf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:53.854156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1238/2439] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:53.861292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e20afa-ee16-4c5b-a850-721ed8137de9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:53.900944 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e20afa-ee16-4c5b-a850-721ed8137de9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:53.908897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18711e7-e650-4c3e-aa01-5ed5d0980b3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:53.909744 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18711e7-e650-4c3e-aa01-5ed5d0980b3f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:53.910035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18711e7-e650-4c3e-aa01-5ed5d0980b3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:53.910352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18711e7-e650-4c3e-aa01-5ed5d0980b3f admin admin] PATH_INFO: `/v3/projects/4f52f8d9a593435da929e9ec5bfbfdd4/users/4f1e83dce28e4ca6b1e5cd237347e572/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:53.936871 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18711e7-e650-4c3e-aa01-5ed5d0980b3f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f52f8d9a593435da929e9ec5bfbfdd4, user_id=4f1e83dce28e4ca6b1e5cd237347e572, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:53.939841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18711e7-e650-4c3e-aa01-5ed5d0980b3f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:53.969484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1202/2440] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:03:53 2022] PUT /identity/v3/projects/4f52f8d9a593435da929e9ec5bfbfdd4/users/4f1e83dce28e4ca6b1e5cd237347e572/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:03:53.973847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d85ea5-95cc-4e7b-9007-d946a2132dbf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:53.988529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d85ea5-95cc-4e7b-9007-d946a2132dbf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:54.080350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08f4d3b-95bb-437f-a6bb-a38d7086846f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:54.080980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f4d3b-95bb-437f-a6bb-a38d7086846f admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:54.081314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f4d3b-95bb-437f-a6bb-a38d7086846f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:54.081590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08f4d3b-95bb-437f-a6bb-a38d7086846f admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:54.086802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f4d3b-95bb-437f-a6bb-a38d7086846f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:54.086802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08f4d3b-95bb-437f-a6bb-a38d7086846f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:54.107433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18711e7-e650-4c3e-aa01-5ed5d0980b3f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:03:54.108195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1239/2441] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:03:53 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:03:54.120591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18711e7-e650-4c3e-aa01-5ed5d0980b3f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:03:54.133051 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e298dc5-d499-4cd5-b216-843656664779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:03:54.133811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e298dc5-d499-4cd5-b216-843656664779 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:03:54.134129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e298dc5-d499-4cd5-b216-843656664779 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:03:54.134416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e298dc5-d499-4cd5-b216-843656664779 admin admin] PATH_INFO: `/v3/projects/4f52f8d9a593435da929e9ec5bfbfdd4/users/4f1e83dce28e4ca6b1e5cd237347e572/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:03:54.136915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e298dc5-d499-4cd5-b216-843656664779 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f52f8d9a593435da929e9ec5bfbfdd4, user_id=4f1e83dce28e4ca6b1e5cd237347e572, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:03:54.140210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e298dc5-d499-4cd5-b216-843656664779 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:03:54.162906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1203/2442] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:03:54 2022] PUT /identity/v3/projects/4f52f8d9a593435da929e9ec5bfbfdd4/users/4f1e83dce28e4ca6b1e5cd237347e572/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:04:00.331832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e50da5-b77c-4868-8041-21d3fbe0dd4d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:00.332477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e50da5-b77c-4868-8041-21d3fbe0dd4d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:00.332477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e50da5-b77c-4868-8041-21d3fbe0dd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:00.380159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e4e50da5-b77c-4868-8041-21d3fbe0dd4d None None] MFA Rules not processed for user `4f1e83dce28e4ca6b1e5cd237347e572`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:00.381644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e4e50da5-b77c-4868-8041-21d3fbe0dd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:00.429043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:00.455475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1240/2443] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:00 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:00.469510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e298dc5-d499-4cd5-b216-843656664779 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:00.472027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8e298dc5-d499-4cd5-b216-843656664779 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:00.554047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e298dc5-d499-4cd5-b216-843656664779 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:00.560393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ab06f7-7fc5-44bb-b130-44e1969e41a3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:00.561022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab06f7-7fc5-44bb-b130-44e1969e41a3 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:00.561298 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab06f7-7fc5-44bb-b130-44e1969e41a3 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:00.561574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ab06f7-7fc5-44bb-b130-44e1969e41a3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:00.615012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:00.631791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab06f7-7fc5-44bb-b130-44e1969e41a3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:00.633409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ab06f7-7fc5-44bb-b130-44e1969e41a3 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:00.642974 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1204/2444] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:02.238271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf70a24-b3f5-48fa-9e07-971465a772ff None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:02.239253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf70a24-b3f5-48fa-9e07-971465a772ff None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:02.239530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf70a24-b3f5-48fa-9e07-971465a772ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:02.293130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ddf70a24-b3f5-48fa-9e07-971465a772ff None None] MFA Rules not processed for user `a04d90d3f1e04a00b2ace1893ce5ef71`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:02.294507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ddf70a24-b3f5-48fa-9e07-971465a772ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:02.314239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1241/2445] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:03.743065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016147d-2312-417a-a258-bf548c92a3a4 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:03.744229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016147d-2312-417a-a258-bf548c92a3a4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:03.745292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e016147d-2312-417a-a258-bf548c92a3a4 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:03.747435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1205/2446] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:04:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:04:03.754143 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0e09a-c7dd-450a-a758-fbfa958a8aec None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:03.755056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0e09a-c7dd-450a-a758-fbfa958a8aec None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:03.755795 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc0e09a-c7dd-450a-a758-fbfa958a8aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:03.781828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9fc0e09a-c7dd-450a-a758-fbfa958a8aec None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:03.784111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9fc0e09a-c7dd-450a-a758-fbfa958a8aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:03.801843 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1242/2447] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:04:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:03.813504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016147d-2312-417a-a258-bf548c92a3a4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:03.815660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e016147d-2312-417a-a258-bf548c92a3a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:03.908426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e016147d-2312-417a-a258-bf548c92a3a4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:03.927568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249de47-849f-4bea-9085-b9d75a717f3c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3505c8965c4a42e493a2f5916cab56dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:03.928425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249de47-849f-4bea-9085-b9d75a717f3c service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:03.928877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249de47-849f-4bea-9085-b9d75a717f3c service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:03.929504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249de47-849f-4bea-9085-b9d75a717f3c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:03.943003 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249de47-849f-4bea-9085-b9d75a717f3c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:03.946814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3249de47-849f-4bea-9085-b9d75a717f3c service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:03.959805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1206/2448] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:04.042104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe2f5-80c3-4a14-992b-cfa8e02b6120 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:04.042417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe2f5-80c3-4a14-992b-cfa8e02b6120 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:04.042896 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45fe2f5-80c3-4a14-992b-cfa8e02b6120 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:04.044273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1243/2449] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:04:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:04:04.049195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5232-0e97-4ffe-9c42-0501afdbfe45 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:04.049552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5232-0e97-4ffe-9c42-0501afdbfe45 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:04.049892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdb5232-0e97-4ffe-9c42-0501afdbfe45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:04.071444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dbdb5232-0e97-4ffe-9c42-0501afdbfe45 None None] MFA Rules not processed for user `f94aa8af227a42a08d8a0e4b7f02b85a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:04.073387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dbdb5232-0e97-4ffe-9c42-0501afdbfe45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:04.136618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1207/2450] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:04:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:04.164392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe2f5-80c3-4a14-992b-cfa8e02b6120 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:04.164392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45fe2f5-80c3-4a14-992b-cfa8e02b6120 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:04.168375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f349ef-13e1-41c6-8f01-659201281480 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:04.169055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f349ef-13e1-41c6-8f01-659201281480 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:04.169425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f349ef-13e1-41c6-8f01-659201281480 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:04.169607 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f349ef-13e1-41c6-8f01-659201281480 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:04.181254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f349ef-13e1-41c6-8f01-659201281480 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:04.181254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f349ef-13e1-41c6-8f01-659201281480 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:04.189608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1244/2451] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:04.297806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141bebf-145c-4e71-a24f-37d2dc68e96f None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:04.298115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141bebf-145c-4e71-a24f-37d2dc68e96f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:04.298401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141bebf-145c-4e71-a24f-37d2dc68e96f None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:04.300110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1208/2452] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:04:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:04:04.305904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1ad8-6c10-4b21-b404-428562ecaeb3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:04.306247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1ad8-6c10-4b21-b404-428562ecaeb3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:04.306572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9de1ad8-6c10-4b21-b404-428562ecaeb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:04.344072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a9de1ad8-6c10-4b21-b404-428562ecaeb3 None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:04.345768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a9de1ad8-6c10-4b21-b404-428562ecaeb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:04.370558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1245/2453] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:04:04 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:04.388741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141bebf-145c-4e71-a24f-37d2dc68e96f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:04.402133 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141bebf-145c-4e71-a24f-37d2dc68e96f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:04.406420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467c3d0-6a48-4e05-92ec-5a4b79564487 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:04.407215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467c3d0-6a48-4e05-92ec-5a4b79564487 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:04.407500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467c3d0-6a48-4e05-92ec-5a4b79564487 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:04.407773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467c3d0-6a48-4e05-92ec-5a4b79564487 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:04.423737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467c3d0-6a48-4e05-92ec-5a4b79564487 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:04.425368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467c3d0-6a48-4e05-92ec-5a4b79564487 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:04.461633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1209/2454] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:09.610151 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de1ad8-6c10-4b21-b404-428562ecaeb3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:09.620205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9de1ad8-6c10-4b21-b404-428562ecaeb3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:09.624034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d011ffef-fb1f-47b7-9b25-1bee7b87689c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:09.624649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011ffef-fb1f-47b7-9b25-1bee7b87689c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:09.625134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011ffef-fb1f-47b7-9b25-1bee7b87689c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:09.625507 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d011ffef-fb1f-47b7-9b25-1bee7b87689c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:09.627270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d011ffef-fb1f-47b7-9b25-1bee7b87689c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:09.697625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011ffef-fb1f-47b7-9b25-1bee7b87689c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:09.699328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d011ffef-fb1f-47b7-9b25-1bee7b87689c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:09.709453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1246/2455] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:09 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:14.432388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=61 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:15.094513 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467c3d0-6a48-4e05-92ec-5a4b79564487 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:15.096389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b467c3d0-6a48-4e05-92ec-5a4b79564487 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:15.197257 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467c3d0-6a48-4e05-92ec-5a4b79564487 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:15.204330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:15.204947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:15.205247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:15.205607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:15.208829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:15.223913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:15.225700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:15.237229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1210/2456] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:16.538152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221630e-6b04-4442-b110-e39c47adc80d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:16.538711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221630e-6b04-4442-b110-e39c47adc80d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:16.538711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1221630e-6b04-4442-b110-e39c47adc80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:16.560412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1221630e-6b04-4442-b110-e39c47adc80d None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:16.561980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1221630e-6b04-4442-b110-e39c47adc80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:16.631124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1247/2457] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:04:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:16.656512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:16.657980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:16.800150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4a398d-982a-4e38-9978-b5fdfa3ad44c service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:16.806167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fcf60-6904-4ac9-9a6a-b1298727579f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:16.807941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fcf60-6904-4ac9-9a6a-b1298727579f service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:16.808437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fcf60-6904-4ac9-9a6a-b1298727579f service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:16.808913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7fcf60-6904-4ac9-9a6a-b1298727579f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:16.832861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fcf60-6904-4ac9-9a6a-b1298727579f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:16.834863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7fcf60-6904-4ac9-9a6a-b1298727579f service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:16.858704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1211/2458] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:04:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:04:17.828471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0510c-2de3-410f-9a0f-39781053dccf None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:17.829099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0510c-2de3-410f-9a0f-39781053dccf None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:17.829099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0510c-2de3-410f-9a0f-39781053dccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:17.856926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-aef0510c-2de3-410f-9a0f-39781053dccf None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:17.858181 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-aef0510c-2de3-410f-9a0f-39781053dccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:17.867215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1248/2459] 10.222.0.25 () {58 vars in 967 bytes} [Tue Sep 13 07:04:17 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:17.881948 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fcf60-6904-4ac9-9a6a-b1298727579f service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:17.893411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7fcf60-6904-4ac9-9a6a-b1298727579f service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:17.900119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ec5c4-b4b7-46be-a958-8805bf94d22e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:17.901165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ec5c4-b4b7-46be-a958-8805bf94d22e service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:17.901448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ec5c4-b4b7-46be-a958-8805bf94d22e service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:17.901724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672ec5c4-b4b7-46be-a958-8805bf94d22e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:17.912814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ec5c4-b4b7-46be-a958-8805bf94d22e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:17.921098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672ec5c4-b4b7-46be-a958-8805bf94d22e service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:17.932628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1212/2460] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:04:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:04:18.255296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026b700-5f35-4adc-bf19-df445fcd1918 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:18.255593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026b700-5f35-4adc-bf19-df445fcd1918 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:18.256161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b026b700-5f35-4adc-bf19-df445fcd1918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:18.278690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b026b700-5f35-4adc-bf19-df445fcd1918 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:18.279903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b026b700-5f35-4adc-bf19-df445fcd1918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:18.288785 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1249/2461] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:04:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:18.307333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ec5c4-b4b7-46be-a958-8805bf94d22e service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:18.321036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672ec5c4-b4b7-46be-a958-8805bf94d22e service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:18.327404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c74fff-e087-4230-a1b4-f36779334c84 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:18.328096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c74fff-e087-4230-a1b4-f36779334c84 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:18.328445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c74fff-e087-4230-a1b4-f36779334c84 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:18.328816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c74fff-e087-4230-a1b4-f36779334c84 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:18.340046 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c74fff-e087-4230-a1b4-f36779334c84 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:18.341756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c74fff-e087-4230-a1b4-f36779334c84 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:18.350951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1213/2462] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:04:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:04:18.708850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fc3b3-526f-4aad-87a9-c88142f27694 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:18.709205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fc3b3-526f-4aad-87a9-c88142f27694 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:18.709547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fc3b3-526f-4aad-87a9-c88142f27694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:18.733622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f82fc3b3-526f-4aad-87a9-c88142f27694 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:18.734886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f82fc3b3-526f-4aad-87a9-c88142f27694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:18.743361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1250/2463] 10.222.0.25 () {58 vars in 967 bytes} [Tue Sep 13 07:04:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:04:18.757584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c74fff-e087-4230-a1b4-f36779334c84 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:18.769308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c74fff-e087-4230-a1b4-f36779334c84 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:18.775497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba101707-b012-4a14-90dc-31b1c9f4c603 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:18.776121 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba101707-b012-4a14-90dc-31b1c9f4c603 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:18.776402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba101707-b012-4a14-90dc-31b1c9f4c603 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:18.776826 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba101707-b012-4a14-90dc-31b1c9f4c603 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:18.785888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba101707-b012-4a14-90dc-31b1c9f4c603 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:18.787828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba101707-b012-4a14-90dc-31b1c9f4c603 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:18.797645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1214/2464] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:04:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:04:18.946921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22b8b7-a534-4566-8d9d-d0ee344103ee None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:18.947434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22b8b7-a534-4566-8d9d-d0ee344103ee None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:18.947889 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd22b8b7-a534-4566-8d9d-d0ee344103ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:18.967859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cd22b8b7-a534-4566-8d9d-d0ee344103ee None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:18.969051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cd22b8b7-a534-4566-8d9d-d0ee344103ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:18.979665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1251/2465] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:04:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:18.991878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba101707-b012-4a14-90dc-31b1c9f4c603 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:19.003138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba101707-b012-4a14-90dc-31b1c9f4c603 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:19.008534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df625e8-8a74-43bb-8e9f-119be24d52ee service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:19.009145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df625e8-8a74-43bb-8e9f-119be24d52ee service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:19.009491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df625e8-8a74-43bb-8e9f-119be24d52ee service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:19.009764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df625e8-8a74-43bb-8e9f-119be24d52ee service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:19.019639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df625e8-8a74-43bb-8e9f-119be24d52ee service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:19.021252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df625e8-8a74-43bb-8e9f-119be24d52ee service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:19.030117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1215/2466] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:04:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:04:21.145335 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22b8b7-a534-4566-8d9d-d0ee344103ee None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:21.146959 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cd22b8b7-a534-4566-8d9d-d0ee344103ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:21.179991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:21.224483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd22b8b7-a534-4566-8d9d-d0ee344103ee None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:21.229465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:21.230219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:21.230668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:21.231121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] PATH_INFO: `/v3/users/7e0774e048684f6b8f79ee44f1c4ff7c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:21.232886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e0774e048684f6b8f79ee44f1c4ff7c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:21.235073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:21.289467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e0774e048684f6b8f79ee44f1c4ff7c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.290146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e0774e048684f6b8f79ee44f1c4ff7c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.302593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e0774e048684f6b8f79ee44f1c4ff7c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.331305 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e0774e048684f6b8f79ee44f1c4ff7c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.332228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e0774e048684f6b8f79ee44f1c4ff7c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.348111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1252/2467] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:04:21 2022] DELETE /identity/v3/users/7e0774e048684f6b8f79ee44f1c4ff7c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:04:21.352511 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba72e1e-38f6-4067-8909-f9446cb95154 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:21.353272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba72e1e-38f6-4067-8909-f9446cb95154 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:21.353272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba72e1e-38f6-4067-8909-f9446cb95154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:21.405369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3ba72e1e-38f6-4067-8909-f9446cb95154 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:21.405369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3ba72e1e-38f6-4067-8909-f9446cb95154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:21.417708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:21.480477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1216/2468] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:21.487296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:21.488902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:21.553149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728e2f95-3dc2-4c6a-bc37-c2a737875316 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:21.557207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98041c10-1686-44a5-8748-5ebcb98e1cde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:21.557837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041c10-1686-44a5-8748-5ebcb98e1cde service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:21.558138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041c10-1686-44a5-8748-5ebcb98e1cde service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:21.558423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98041c10-1686-44a5-8748-5ebcb98e1cde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:21.568254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98041c10-1686-44a5-8748-5ebcb98e1cde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:21.569757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98041c10-1686-44a5-8748-5ebcb98e1cde service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:21.580983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1253/2469] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:21.741781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba72e1e-38f6-4067-8909-f9446cb95154 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:21.745064 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3ba72e1e-38f6-4067-8909-f9446cb95154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:21.769223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba72e1e-38f6-4067-8909-f9446cb95154 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:21.774364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:21.775863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:21.775863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:21.776064 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] PATH_INFO: `/v3/users/3684e8422d6b4c53836519016dc0e8da` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:21.777386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] RBAC: Authorizing `identity:delete_user(user_id=3684e8422d6b4c53836519016dc0e8da)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:21.779764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:21.785249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98041c10-1686-44a5-8748-5ebcb98e1cde service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:21.787079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-98041c10-1686-44a5-8748-5ebcb98e1cde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:21.834921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98041c10-1686-44a5-8748-5ebcb98e1cde service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:21.840824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:21.842192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:21.842554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:21.842828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] PATH_INFO: `/v3/projects/5006f7a1770f4d9a96cbec0715987237` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:21.844773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5006f7a1770f4d9a96cbec0715987237)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:21.847105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:21.868104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3684e8422d6b4c53836519016dc0e8da'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.887361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3684e8422d6b4c53836519016dc0e8da'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.918016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3684e8422d6b4c53836519016dc0e8da'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.918813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3684e8422d6b4c53836519016dc0e8da'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.920807 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:21.926782 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3c76632f-b55d-47c1-9233-d72ccb117fab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3684e8422d6b4c53836519016dc0e8da'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.928729 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5006f7a1770f4d9a96cbec0715987237', 'request_id': 'req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.933254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1217/2470] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:04:21 2022] DELETE /identity/v3/users/3684e8422d6b4c53836519016dc0e8da => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:04:21.936709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5006f7a1770f4d9a96cbec0715987237', 'request_id': 'req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.937665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64387437-68c0-454c-a762-95f2018c0de8 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:21.938156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64387437-68c0-454c-a762-95f2018c0de8 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:21.938440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64387437-68c0-454c-a762-95f2018c0de8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:21.943267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5006f7a1770f4d9a96cbec0715987237', 'request_id': 'req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.962292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-64387437-68c0-454c-a762-95f2018c0de8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:21.963482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] The token cache is being invalidate because project 5006f7a1770f4d9a96cbec0715987237 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:04:21.963824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.964432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-64387437-68c0-454c-a762-95f2018c0de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:21.964571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:21.966123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1254/2471] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:04:21 2022] DELETE /identity/v3/projects/5006f7a1770f4d9a96cbec0715987237 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:04:22.069406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1218/2472] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:04:22.079899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:22.081582 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:22.214451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9258bd37-b2de-41a6-9d3a-0ca4e79d63bc admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:22.281472 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e27807-7999-49a1-82d2-25683c0d17e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:22.282732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27807-7999-49a1-82d2-25683c0d17e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:22.283240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27807-7999-49a1-82d2-25683c0d17e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:22.283747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e27807-7999-49a1-82d2-25683c0d17e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:22.294877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27807-7999-49a1-82d2-25683c0d17e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:22.296624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e27807-7999-49a1-82d2-25683c0d17e0 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:22.307383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1255/2473] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:22.643402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64387437-68c0-454c-a762-95f2018c0de8 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:22.645909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-64387437-68c0-454c-a762-95f2018c0de8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:22.663700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64387437-68c0-454c-a762-95f2018c0de8 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:22.679310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:22.679992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:22.680272 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:22.680544 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] PATH_INFO: `/v3/projects/21a4c6cf46844a269832e0b6f919ebf9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:22.682270 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=21a4c6cf46844a269832e0b6f919ebf9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:22.684230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:22.731836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:22.759975 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21a4c6cf46844a269832e0b6f919ebf9', 'request_id': 'req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:22.768700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21a4c6cf46844a269832e0b6f919ebf9', 'request_id': 'req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:22.779562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21a4c6cf46844a269832e0b6f919ebf9', 'request_id': 'req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:22.797086 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] The token cache is being invalidate because project 21a4c6cf46844a269832e0b6f919ebf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:04:22.797364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:22.798363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-185d6e27-f454-4eea-9b2f-2c2fdaded3dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:04:22.799990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1219/2474] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:04:22 2022] DELETE /identity/v3/projects/21a4c6cf46844a269832e0b6f919ebf9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:04:22.807555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bd7a2-754d-410d-ab74-f91dd076ff72 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:22.808115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bd7a2-754d-410d-ab74-f91dd076ff72 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:22.808554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990bd7a2-754d-410d-ab74-f91dd076ff72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:22.831715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-990bd7a2-754d-410d-ab74-f91dd076ff72 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:22.833100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-990bd7a2-754d-410d-ab74-f91dd076ff72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:22.901449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1256/2475] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:22.973541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f704f0b-9c53-43a9-ae72-f9052660605e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:22.973819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f704f0b-9c53-43a9-ae72-f9052660605e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:22.974107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f704f0b-9c53-43a9-ae72-f9052660605e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:22.995606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-0f704f0b-9c53-43a9-ae72-f9052660605e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:22.996740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0f704f0b-9c53-43a9-ae72-f9052660605e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:23.004806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1220/2476] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:23.009694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bd7a2-754d-410d-ab74-f91dd076ff72 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:23.023271 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990bd7a2-754d-410d-ab74-f91dd076ff72 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:23.029409 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738615d-be62-4db0-b22e-b911ff9c59cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:23.030036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738615d-be62-4db0-b22e-b911ff9c59cd admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:23.030348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738615d-be62-4db0-b22e-b911ff9c59cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:23.030739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c738615d-be62-4db0-b22e-b911ff9c59cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:23.031548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738615d-be62-4db0-b22e-b911ff9c59cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:04:23.031859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738615d-be62-4db0-b22e-b911ff9c59cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:23.033489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c738615d-be62-4db0-b22e-b911ff9c59cd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:23.048102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1257/2477] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:04:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:04:23.052895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f704f0b-9c53-43a9-ae72-f9052660605e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:23.063130 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f704f0b-9c53-43a9-ae72-f9052660605e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:23.079247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780cedf-3a04-4270-bc79-590a99dac614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:23.080192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780cedf-3a04-4270-bc79-590a99dac614 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:23.081865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780cedf-3a04-4270-bc79-590a99dac614 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:23.081865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f780cedf-3a04-4270-bc79-590a99dac614 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:23.081865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780cedf-3a04-4270-bc79-590a99dac614 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:23.083797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f780cedf-3a04-4270-bc79-590a99dac614 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:23.136324 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1221/2478] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:04:23 2022] POST /identity/v3/projects => generated 379 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:04:23.140884 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738615d-be62-4db0-b22e-b911ff9c59cd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:23.152115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c738615d-be62-4db0-b22e-b911ff9c59cd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:23.163501 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:23.313461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46724b8-4ac7-4099-951d-80f74605a3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:23.315208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46724b8-4ac7-4099-951d-80f74605a3dd admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:23.315667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46724b8-4ac7-4099-951d-80f74605a3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:23.316103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46724b8-4ac7-4099-951d-80f74605a3dd admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:23.317011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46724b8-4ac7-4099-951d-80f74605a3dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:23.319268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46724b8-4ac7-4099-951d-80f74605a3dd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:23.384200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1258/2479] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:04:23 2022] POST /identity/v3/users => generated 290 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:04:23.386677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780cedf-3a04-4270-bc79-590a99dac614 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:23.404562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f780cedf-3a04-4270-bc79-590a99dac614 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:23.418670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a861288-d398-49ef-be74-9fc475e78147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:23.419308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a861288-d398-49ef-be74-9fc475e78147 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:23.419579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a861288-d398-49ef-be74-9fc475e78147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:23.419842 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a861288-d398-49ef-be74-9fc475e78147 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:23.420487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a861288-d398-49ef-be74-9fc475e78147 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:23.422146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a861288-d398-49ef-be74-9fc475e78147 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:23.444370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1222/2480] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:04:23.449231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46724b8-4ac7-4099-951d-80f74605a3dd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:23.478430 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46724b8-4ac7-4099-951d-80f74605a3dd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:23.485504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee7e13-c291-41d1-ae2a-b1d9bf687468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:23.486465 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee7e13-c291-41d1-ae2a-b1d9bf687468 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:23.487079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee7e13-c291-41d1-ae2a-b1d9bf687468 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:23.487355 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee7e13-c291-41d1-ae2a-b1d9bf687468 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:23.488012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee7e13-c291-41d1-ae2a-b1d9bf687468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:23.489669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ee7e13-c291-41d1-ae2a-b1d9bf687468 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:23.510455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1259/2481] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:04:23 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:04:23.513210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a861288-d398-49ef-be74-9fc475e78147 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:23.534362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a861288-d398-49ef-be74-9fc475e78147 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:23.551884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f16b4b-6c43-4ea6-98e8-812274c2e707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:23.557950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f16b4b-6c43-4ea6-98e8-812274c2e707 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:23.558314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f16b4b-6c43-4ea6-98e8-812274c2e707 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:23.558957 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f16b4b-6c43-4ea6-98e8-812274c2e707 admin admin] PATH_INFO: `/v3/projects/99bbe5cbd59f4feda4da45f2b14bd09b/users/67fa873d16414749bc478a59de3ec5b4/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:23.581572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f16b4b-6c43-4ea6-98e8-812274c2e707 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99bbe5cbd59f4feda4da45f2b14bd09b, user_id=67fa873d16414749bc478a59de3ec5b4, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:23.584582 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f16b4b-6c43-4ea6-98e8-812274c2e707 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:23.620067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1223/2482] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:04:23 2022] PUT /identity/v3/projects/99bbe5cbd59f4feda4da45f2b14bd09b/users/67fa873d16414749bc478a59de3ec5b4/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:04:23.625521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107762f7-5ed5-4f3b-a16a-d1ce54a3c517 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:23.625937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107762f7-5ed5-4f3b-a16a-d1ce54a3c517 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:23.626282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107762f7-5ed5-4f3b-a16a-d1ce54a3c517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:23.657874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-107762f7-5ed5-4f3b-a16a-d1ce54a3c517 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:23.659205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-107762f7-5ed5-4f3b-a16a-d1ce54a3c517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:23.741486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1260/2483] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:23.749910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f16b4b-6c43-4ea6-98e8-812274c2e707 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:23.752433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-55f16b4b-6c43-4ea6-98e8-812274c2e707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:23.828595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f16b4b-6c43-4ea6-98e8-812274c2e707 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:23.832371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0328911-5ab8-46a9-9472-f5204f79edd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:23.833091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0328911-5ab8-46a9-9472-f5204f79edd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:23.833461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0328911-5ab8-46a9-9472-f5204f79edd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:23.833850 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0328911-5ab8-46a9-9472-f5204f79edd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:23.845890 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0328911-5ab8-46a9-9472-f5204f79edd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:23.847413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0328911-5ab8-46a9-9472-f5204f79edd0 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:23.863031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1224/2484] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:29.234005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57a9f5-8153-4cfc-8ca7-d20631d60076 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:29.234592 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57a9f5-8153-4cfc-8ca7-d20631d60076 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:29.234747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57a9f5-8153-4cfc-8ca7-d20631d60076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:29.266805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1a57a9f5-8153-4cfc-8ca7-d20631d60076 None None] MFA Rules not processed for user `67fa873d16414749bc478a59de3ec5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:29.268047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1a57a9f5-8153-4cfc-8ca7-d20631d60076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:29.336634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1261/2485] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:29 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:29.341486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0328911-5ab8-46a9-9472-f5204f79edd0 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:29.351639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0328911-5ab8-46a9-9472-f5204f79edd0 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:29.356372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56eb188-f235-4394-8242-aaee12fc47ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:29.356975 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56eb188-f235-4394-8242-aaee12fc47ab admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:29.357249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56eb188-f235-4394-8242-aaee12fc47ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:29.357520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56eb188-f235-4394-8242-aaee12fc47ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:29.358180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56eb188-f235-4394-8242-aaee12fc47ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:29.360122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56eb188-f235-4394-8242-aaee12fc47ab admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:29.391315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1225/2486] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:04:29 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:04:29.401442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57a9f5-8153-4cfc-8ca7-d20631d60076 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:29.414493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57a9f5-8153-4cfc-8ca7-d20631d60076 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:29.437934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:29.481333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3dd51-b88d-4011-b338-dabb4a59aab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:29.481957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3dd51-b88d-4011-b338-dabb4a59aab7 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:29.482414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3dd51-b88d-4011-b338-dabb4a59aab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:29.482746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d3dd51-b88d-4011-b338-dabb4a59aab7 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:29.483469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3dd51-b88d-4011-b338-dabb4a59aab7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:29.485479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d3dd51-b88d-4011-b338-dabb4a59aab7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:29.509387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1262/2487] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:04:29 2022] POST /identity/v3/users => generated 295 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:04:29.513403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56eb188-f235-4394-8242-aaee12fc47ab admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:29.524129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56eb188-f235-4394-8242-aaee12fc47ab admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:29.528921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75391d41-507c-4d2f-be0c-4bfdc615965e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:29.529549 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75391d41-507c-4d2f-be0c-4bfdc615965e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:29.529993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75391d41-507c-4d2f-be0c-4bfdc615965e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:29.530746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75391d41-507c-4d2f-be0c-4bfdc615965e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:29.531462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75391d41-507c-4d2f-be0c-4bfdc615965e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:29.532987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75391d41-507c-4d2f-be0c-4bfdc615965e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:29.545099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1226/2488] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:04:29.549719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3dd51-b88d-4011-b338-dabb4a59aab7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:29.561674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d3dd51-b88d-4011-b338-dabb4a59aab7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:29.566575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5aefc-aff6-4097-8302-652ea3d655d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:29.567257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5aefc-aff6-4097-8302-652ea3d655d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:29.567539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5aefc-aff6-4097-8302-652ea3d655d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:29.567812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb5aefc-aff6-4097-8302-652ea3d655d1 admin admin] PATH_INFO: `/v3/projects/6eae938afce74856b6dfb2dede2b59cc/users/f5d33a38add34abaa240270dc11af380/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:29.585854 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5aefc-aff6-4097-8302-652ea3d655d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eae938afce74856b6dfb2dede2b59cc, user_id=f5d33a38add34abaa240270dc11af380, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:29.588990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb5aefc-aff6-4097-8302-652ea3d655d1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:29.605915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1263/2489] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:04:29 2022] PUT /identity/v3/projects/6eae938afce74856b6dfb2dede2b59cc/users/f5d33a38add34abaa240270dc11af380/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:04:29.609776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75391d41-507c-4d2f-be0c-4bfdc615965e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:29.620166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75391d41-507c-4d2f-be0c-4bfdc615965e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:29.654848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:29.691396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab826c9-38d5-4d3e-a5f9-90dc9235b7f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:29.692032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab826c9-38d5-4d3e-a5f9-90dc9235b7f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:29.692315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab826c9-38d5-4d3e-a5f9-90dc9235b7f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:29.692587 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab826c9-38d5-4d3e-a5f9-90dc9235b7f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:29.693235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab826c9-38d5-4d3e-a5f9-90dc9235b7f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:29.694940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab826c9-38d5-4d3e-a5f9-90dc9235b7f2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:29.705910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1227/2490] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:04:29 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:04:29.710247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5aefc-aff6-4097-8302-652ea3d655d1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:29.721728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb5aefc-aff6-4097-8302-652ea3d655d1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:29.726574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e427b-0258-4f1b-a404-ac41ae8f0010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:29.727319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e427b-0258-4f1b-a404-ac41ae8f0010 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:29.727695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e427b-0258-4f1b-a404-ac41ae8f0010 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:29.728085 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8e427b-0258-4f1b-a404-ac41ae8f0010 admin admin] PATH_INFO: `/v3/projects/6eae938afce74856b6dfb2dede2b59cc/users/f5d33a38add34abaa240270dc11af380/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:29.730107 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e427b-0258-4f1b-a404-ac41ae8f0010 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eae938afce74856b6dfb2dede2b59cc, user_id=f5d33a38add34abaa240270dc11af380, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:29.732907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8e427b-0258-4f1b-a404-ac41ae8f0010 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:29.749254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1264/2491] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:04:29 2022] PUT /identity/v3/projects/6eae938afce74856b6dfb2dede2b59cc/users/f5d33a38add34abaa240270dc11af380/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:04:36.111148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33ca4-ed76-41ac-a6bf-eaa43f028444 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:36.112733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33ca4-ed76-41ac-a6bf-eaa43f028444 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:36.113014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a33ca4-ed76-41ac-a6bf-eaa43f028444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:36.153628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-18a33ca4-ed76-41ac-a6bf-eaa43f028444 None None] MFA Rules not processed for user `f5d33a38add34abaa240270dc11af380`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:36.154942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-18a33ca4-ed76-41ac-a6bf-eaa43f028444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:36.238419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1228/2492] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:36 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:36.253740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e427b-0258-4f1b-a404-ac41ae8f0010 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:36.255749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2b8e427b-0258-4f1b-a404-ac41ae8f0010 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:36.349781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8e427b-0258-4f1b-a404-ac41ae8f0010 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:36.355563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac961c2-672a-4735-a89b-a0e5a4aab08f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:36.356198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac961c2-672a-4735-a89b-a0e5a4aab08f service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:36.356478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac961c2-672a-4735-a89b-a0e5a4aab08f service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:36.356756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac961c2-672a-4735-a89b-a0e5a4aab08f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:36.436381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac961c2-672a-4735-a89b-a0e5a4aab08f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:36.438197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac961c2-672a-4735-a89b-a0e5a4aab08f service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:36.450117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1265/2493] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:43.061892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66890375-0cef-4eb0-b26a-ae67834e7866 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:43.062537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66890375-0cef-4eb0-b26a-ae67834e7866 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:43.062537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66890375-0cef-4eb0-b26a-ae67834e7866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:43.083250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-66890375-0cef-4eb0-b26a-ae67834e7866 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:43.084535 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-66890375-0cef-4eb0-b26a-ae67834e7866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:43.152531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1229/2494] 10.222.0.25 () {58 vars in 967 bytes} [Tue Sep 13 07:04:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:43.166563 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac961c2-672a-4735-a89b-a0e5a4aab08f service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:43.168149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6ac961c2-672a-4735-a89b-a0e5a4aab08f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:43.183218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:43.255196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac961c2-672a-4735-a89b-a0e5a4aab08f service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:43.261041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167a7dc9-caaf-4e9a-8953-d1d52dee2fba service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:43.261690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a7dc9-caaf-4e9a-8953-d1d52dee2fba service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:43.261965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a7dc9-caaf-4e9a-8953-d1d52dee2fba service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:43.262270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167a7dc9-caaf-4e9a-8953-d1d52dee2fba service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:43.271272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a7dc9-caaf-4e9a-8953-d1d52dee2fba service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:43.272862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167a7dc9-caaf-4e9a-8953-d1d52dee2fba service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:43.281460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1266/2495] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:04:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:04:51.156119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66890375-0cef-4eb0-b26a-ae67834e7866 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:04:51.216275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66890375-0cef-4eb0-b26a-ae67834e7866 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:04:51.219863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098cbb39-c88d-45f8-bcf7-3abc62749f48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:04:51.220623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cbb39-c88d-45f8-bcf7-3abc62749f48 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:51.220901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cbb39-c88d-45f8-bcf7-3abc62749f48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:51.221170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098cbb39-c88d-45f8-bcf7-3abc62749f48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:51.222926 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-098cbb39-c88d-45f8-bcf7-3abc62749f48 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:51.232290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:51.286174 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098cbb39-c88d-45f8-bcf7-3abc62749f48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:04:51.287531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098cbb39-c88d-45f8-bcf7-3abc62749f48 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:04:51.298753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1230/2496] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:04:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:04:53.278183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=27 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:04:54.208364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076cd2f-ee4d-4d05-a96e-56fd842dc471 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:54.208649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076cd2f-ee4d-4d05-a96e-56fd842dc471 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:54.208922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d076cd2f-ee4d-4d05-a96e-56fd842dc471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:54.229557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d076cd2f-ee4d-4d05-a96e-56fd842dc471 None None] MFA Rules not processed for user `5a0fd7347cad41bc80f427f872cf78fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:54.231296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d076cd2f-ee4d-4d05-a96e-56fd842dc471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:54.313471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1267/2497] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:04:55.737417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09475a5-886e-4916-a41b-e7a706d4a2bf None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:04:55.737949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09475a5-886e-4916-a41b-e7a706d4a2bf None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:04:55.738248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09475a5-886e-4916-a41b-e7a706d4a2bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:04:55.758878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f09475a5-886e-4916-a41b-e7a706d4a2bf None None] MFA Rules not processed for user `67fa873d16414749bc478a59de3ec5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:04:55.760152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f09475a5-886e-4916-a41b-e7a706d4a2bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:04:55.767293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1231/2498] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:04:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:05:04.306192 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=25 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:05:05.752931 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=28 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:05:13.824567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076cd2f-ee4d-4d05-a96e-56fd842dc471 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:05:13.826246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d076cd2f-ee4d-4d05-a96e-56fd842dc471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:05:13.839806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d076cd2f-ee4d-4d05-a96e-56fd842dc471 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:05:13.843781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ffd4c-a9dd-4ae9-afd8-9c827602eff5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:05:13.844393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ffd4c-a9dd-4ae9-afd8-9c827602eff5 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:05:13.844667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ffd4c-a9dd-4ae9-afd8-9c827602eff5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:05:13.844935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ffd4c-a9dd-4ae9-afd8-9c827602eff5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:05:13.854813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ffd4c-a9dd-4ae9-afd8-9c827602eff5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:05:13.856268 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9ffd4c-a9dd-4ae9-afd8-9c827602eff5 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:05:13.866106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1268/2499] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:05:13 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:05:23.861796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:05:42.884263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aebff6-c258-4b3b-99f4-c4df1c82dcd6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:05:42.884882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aebff6-c258-4b3b-99f4-c4df1c82dcd6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:05:42.884882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aebff6-c258-4b3b-99f4-c4df1c82dcd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:05:42.905001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f5aebff6-c258-4b3b-99f4-c4df1c82dcd6 None None] MFA Rules not processed for user `0d8d0ff010c04d65b36ebe65c1cbbbfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:05:42.906325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f5aebff6-c258-4b3b-99f4-c4df1c82dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:05:42.964623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1232/2500] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:05:42 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:05:46.554533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced8714-7f38-42d0-9525-7b4eeeb886eb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:05:46.555110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced8714-7f38-42d0-9525-7b4eeeb886eb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:05:46.555224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ced8714-7f38-42d0-9525-7b4eeeb886eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:05:46.581095 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5ced8714-7f38-42d0-9525-7b4eeeb886eb None None] MFA Rules not processed for user `ecf2dfd9bbe54b3085e9638caf8ad044`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:05:46.582346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5ced8714-7f38-42d0-9525-7b4eeeb886eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:05:46.640924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1269/2501] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:05:46 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:05:52.957425 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=25 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:05:56.633172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=25 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:05.816233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aebff6-c258-4b3b-99f4-c4df1c82dcd6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:05.818451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f5aebff6-c258-4b3b-99f4-c4df1c82dcd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:05.883232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aebff6-c258-4b3b-99f4-c4df1c82dcd6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:05.887399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9568fd-c41f-44f1-aa29-4b7de1e367e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:05.888028 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9568fd-c41f-44f1-aa29-4b7de1e367e0 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:05.888311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9568fd-c41f-44f1-aa29-4b7de1e367e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:05.888586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9568fd-c41f-44f1-aa29-4b7de1e367e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:05.890652 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0f9568fd-c41f-44f1-aa29-4b7de1e367e0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:05.906270 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9568fd-c41f-44f1-aa29-4b7de1e367e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:05.908507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f9568fd-c41f-44f1-aa29-4b7de1e367e0 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:05.924006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1233/2502] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:06.143524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced8714-7f38-42d0-9525-7b4eeeb886eb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:06.158270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ced8714-7f38-42d0-9525-7b4eeeb886eb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:06.162191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ae664-22ca-408e-bc8e-3875a23bfe5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:06.162812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ae664-22ca-408e-bc8e-3875a23bfe5d service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:06.163072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ae664-22ca-408e-bc8e-3875a23bfe5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:06.163344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ae664-22ca-408e-bc8e-3875a23bfe5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:06.165407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f17ae664-22ca-408e-bc8e-3875a23bfe5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:06.179826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ae664-22ca-408e-bc8e-3875a23bfe5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:06.181598 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ae664-22ca-408e-bc8e-3875a23bfe5d service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:06.195933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1270/2503] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:07.057687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9568fd-c41f-44f1-aa29-4b7de1e367e0 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:07.067165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9568fd-c41f-44f1-aa29-4b7de1e367e0 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:07.070997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b49649-7b8f-472c-a88a-7734e8add345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:07.071595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b49649-7b8f-472c-a88a-7734e8add345 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:07.071866 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b49649-7b8f-472c-a88a-7734e8add345 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:07.072135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b49649-7b8f-472c-a88a-7734e8add345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:07.073938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e2b49649-7b8f-472c-a88a-7734e8add345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:07.084806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b49649-7b8f-472c-a88a-7734e8add345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:07.086266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b49649-7b8f-472c-a88a-7734e8add345 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:07.095131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1234/2504] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:14.755892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9c7c3-5445-4e43-8313-813788463ef1 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:14.756399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9c7c3-5445-4e43-8313-813788463ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:14.756530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea9c7c3-5445-4e43-8313-813788463ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:14.776750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-fea9c7c3-5445-4e43-8313-813788463ef1 None None] MFA Rules not processed for user `5a0fd7347cad41bc80f427f872cf78fa`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:06:14.778127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fea9c7c3-5445-4e43-8313-813788463ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:14.786432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1271/2505] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:06:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:06:15.178721 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b49649-7b8f-472c-a88a-7734e8add345 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:15.188684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b49649-7b8f-472c-a88a-7734e8add345 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:15.192657 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93486a95-e1db-4b94-9032-d622283450a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:15.193252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93486a95-e1db-4b94-9032-d622283450a7 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:15.193525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93486a95-e1db-4b94-9032-d622283450a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:15.193792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93486a95-e1db-4b94-9032-d622283450a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:15.195682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-93486a95-e1db-4b94-9032-d622283450a7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:15.205340 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93486a95-e1db-4b94-9032-d622283450a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:15.206828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93486a95-e1db-4b94-9032-d622283450a7 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:15.214762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1235/2506] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:15.550009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9c7c3-5445-4e43-8313-813788463ef1 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:15.551623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fea9c7c3-5445-4e43-8313-813788463ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:15.627868 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea9c7c3-5445-4e43-8313-813788463ef1 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:15.633051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd51768-1001-4940-b92b-e38fbe0f73e5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:15.633661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd51768-1001-4940-b92b-e38fbe0f73e5 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:15.633934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd51768-1001-4940-b92b-e38fbe0f73e5 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:15.634235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd51768-1001-4940-b92b-e38fbe0f73e5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:15.636060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-fdd51768-1001-4940-b92b-e38fbe0f73e5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:15.651524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd51768-1001-4940-b92b-e38fbe0f73e5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:15.653094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd51768-1001-4940-b92b-e38fbe0f73e5 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:15.663492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1272/2507] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:20.964556 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93486a95-e1db-4b94-9032-d622283450a7 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:20.975699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93486a95-e1db-4b94-9032-d622283450a7 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:20.981611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021ea0b-79b9-40f0-8a0e-f5c9e4dca573 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:20.982655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021ea0b-79b9-40f0-8a0e-f5c9e4dca573 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:20.983194 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021ea0b-79b9-40f0-8a0e-f5c9e4dca573 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:20.983727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6021ea0b-79b9-40f0-8a0e-f5c9e4dca573 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:20.986200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6021ea0b-79b9-40f0-8a0e-f5c9e4dca573 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:20.998742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021ea0b-79b9-40f0-8a0e-f5c9e4dca573 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:21.000268 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6021ea0b-79b9-40f0-8a0e-f5c9e4dca573 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:21.009792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1236/2508] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:20 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:21.025543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d35dee-83bb-4e14-8adb-9d882c8d1d9d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:21.026463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d35dee-83bb-4e14-8adb-9d882c8d1d9d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:21.026463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d35dee-83bb-4e14-8adb-9d882c8d1d9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:21.047581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a9d35dee-83bb-4e14-8adb-9d882c8d1d9d None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:06:21.048849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a9d35dee-83bb-4e14-8adb-9d882c8d1d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:21.057743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1273/2509] 10.222.0.25 () {58 vars in 967 bytes} [Tue Sep 13 07:06:21 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:06:21.113091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021ea0b-79b9-40f0-8a0e-f5c9e4dca573 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:21.125764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6021ea0b-79b9-40f0-8a0e-f5c9e4dca573 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:21.139452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1216-5b54-49ed-9c65-ddfe5a6e5467 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:21.140084 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1216-5b54-49ed-9c65-ddfe5a6e5467 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:21.140359 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1216-5b54-49ed-9c65-ddfe5a6e5467 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:21.140649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4b1216-5b54-49ed-9c65-ddfe5a6e5467 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:21.151539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b1216-5b54-49ed-9c65-ddfe5a6e5467 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:21.153910 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4b1216-5b54-49ed-9c65-ddfe5a6e5467 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:21.165563 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1237/2510] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:06:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:06:27.262494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d35dee-83bb-4e14-8adb-9d882c8d1d9d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:27.264286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a9d35dee-83bb-4e14-8adb-9d882c8d1d9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:27.271990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=45 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:27.279396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d35dee-83bb-4e14-8adb-9d882c8d1d9d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:27.283846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d91c1-0cc8-4a40-97ca-5903029c90a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:27.284859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d91c1-0cc8-4a40-97ca-5903029c90a4 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:27.285447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d91c1-0cc8-4a40-97ca-5903029c90a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:27.293141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d91c1-0cc8-4a40-97ca-5903029c90a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:27.296594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b83d91c1-0cc8-4a40-97ca-5903029c90a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:27.311131 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d91c1-0cc8-4a40-97ca-5903029c90a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:27.312775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d91c1-0cc8-4a40-97ca-5903029c90a4 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:27.326542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1274/2511] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:27 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:28.765258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1216-5b54-49ed-9c65-ddfe5a6e5467 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:28.767344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fe4b1216-5b54-49ed-9c65-ddfe5a6e5467 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:28.838894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4b1216-5b54-49ed-9c65-ddfe5a6e5467 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:28.844371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:28.845036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:28.845373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:28.845673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] PATH_INFO: `/v3/users/baa15cc84259470f93afcfe23ef55c2f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:28.846942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=baa15cc84259470f93afcfe23ef55c2f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:28.849026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:28.901854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baa15cc84259470f93afcfe23ef55c2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:28.905494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:28.911759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baa15cc84259470f93afcfe23ef55c2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:28.925134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baa15cc84259470f93afcfe23ef55c2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:28.925870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baa15cc84259470f93afcfe23ef55c2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:28.933349 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baa15cc84259470f93afcfe23ef55c2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:28.939520 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1238/2512] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:06:28 2022] DELETE /identity/v3/users/baa15cc84259470f93afcfe23ef55c2f => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:06:29.126929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d91c1-0cc8-4a40-97ca-5903029c90a4 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:29.140399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d91c1-0cc8-4a40-97ca-5903029c90a4 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:29.214384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:29.215042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:29.215321 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:29.215596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] PATH_INFO: `/v3/projects/76569a470885443abfbe4e14e9b2f505` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:29.216910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=76569a470885443abfbe4e14e9b2f505)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:29.218891 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:29.271763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76569a470885443abfbe4e14e9b2f505', 'request_id': 'req-751cc05e-931a-4970-88d8-e96beef5dc5f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.278299 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76569a470885443abfbe4e14e9b2f505', 'request_id': 'req-751cc05e-931a-4970-88d8-e96beef5dc5f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.284460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76569a470885443abfbe4e14e9b2f505', 'request_id': 'req-751cc05e-931a-4970-88d8-e96beef5dc5f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.298754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] The token cache is being invalidate because project 76569a470885443abfbe4e14e9b2f505 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:06:29.299041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.299675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.301303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1275/2513] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:06:29 2022] DELETE /identity/v3/projects/76569a470885443abfbe4e14e9b2f505 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:06:29.305328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:29.307167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:29.320481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1ae2-d474-4609-8dec-bc2777505b41 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:29.379886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:29.380535 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:29.380817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:29.381096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] PATH_INFO: `/v3/users/2e34c55dc58a44ada90803e5530ab447` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:29.382240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e34c55dc58a44ada90803e5530ab447)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:29.384176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:29.435434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e34c55dc58a44ada90803e5530ab447'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.444213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e34c55dc58a44ada90803e5530ab447'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.456967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e34c55dc58a44ada90803e5530ab447'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.457648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e34c55dc58a44ada90803e5530ab447'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.464507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e34c55dc58a44ada90803e5530ab447'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.470388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1239/2514] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:06:29 2022] DELETE /identity/v3/users/2e34c55dc58a44ada90803e5530ab447 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:06:29.647916 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:29.649392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:29.665484 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cc05e-931a-4970-88d8-e96beef5dc5f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:29.733509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:29.734191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:29.734469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:29.734873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] PATH_INFO: `/v3/projects/f866d57a23e44b9ca8a9349a95bc6020` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:29.736002 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f866d57a23e44b9ca8a9349a95bc6020)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:29.738228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:29.762975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:29.795692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f866d57a23e44b9ca8a9349a95bc6020', 'request_id': 'req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.803381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f866d57a23e44b9ca8a9349a95bc6020', 'request_id': 'req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.810023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f866d57a23e44b9ca8a9349a95bc6020', 'request_id': 'req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.827240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] The token cache is being invalidate because project f866d57a23e44b9ca8a9349a95bc6020 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:06:29.827538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.828452 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:29.830548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1276/2515] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:06:29 2022] DELETE /identity/v3/projects/f866d57a23e44b9ca8a9349a95bc6020 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:06:30.285763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:30.287314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:30.349016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:30.378662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096e8dd3-3dac-47b5-a24c-83a39ce2bcd2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:30.445797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:30.446452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:30.446728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:30.446996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:30.448773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:30.460709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:30.462356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:30.473088 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1240/2516] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:06:39.824011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:DELETE=7,keystone:SELECT=15,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:40.468448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=40 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:43.278043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:43.279731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:43.298055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bf8cdc-2c49-49e5-9710-3940be6ed3e0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:43.304895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:43.305818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:43.306122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:43.306408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:43.308330 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:43.381521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:43.383194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:43.392880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1277/2517] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:48.990996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:48.992978 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:49.056231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c899987b-9d16-48c9-9166-ce13708a8ca7 service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:49.060578 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fe994-e6ad-4053-941f-ad0cdb7dca1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:49.061193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fe994-e6ad-4053-941f-ad0cdb7dca1a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:49.061472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fe994-e6ad-4053-941f-ad0cdb7dca1a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:49.061741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9fe994-e6ad-4053-941f-ad0cdb7dca1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:49.064030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4e9fe994-e6ad-4053-941f-ad0cdb7dca1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:49.133339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fe994-e6ad-4053-941f-ad0cdb7dca1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:49.134797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9fe994-e6ad-4053-941f-ad0cdb7dca1a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:49.144458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1241/2518] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:52.485894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:52.488034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:52.502969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a788b-38a4-4d89-8500-eeaaa690161b service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:52.507897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:52.508483 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:52.508761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:52.509039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] PATH_INFO: `/v3/users/0d8d0ff010c04d65b36ebe65c1cbbbfd` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:52.510255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d8d0ff010c04d65b36ebe65c1cbbbfd)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:52.512212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:52.562000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d8d0ff010c04d65b36ebe65c1cbbbfd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.562703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d8d0ff010c04d65b36ebe65c1cbbbfd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.572198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d8d0ff010c04d65b36ebe65c1cbbbfd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.585369 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d8d0ff010c04d65b36ebe65c1cbbbfd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.585984 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-df75ae53-66f3-4551-9dd9-b0f42c3f5b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d8d0ff010c04d65b36ebe65c1cbbbfd'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.599445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1278/2519] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:06:52 2022] DELETE /identity/v3/users/0d8d0ff010c04d65b36ebe65c1cbbbfd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:06:52.781457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fe994-e6ad-4053-941f-ad0cdb7dca1a service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:52.792112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9fe994-e6ad-4053-941f-ad0cdb7dca1a service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:52.852517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:52.853162 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:52.853442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:52.853714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] PATH_INFO: `/v3/projects/38b6d7b146a34896bb8309e2c51348f8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:52.854852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38b6d7b146a34896bb8309e2c51348f8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:52.856778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:52.905906 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38b6d7b146a34896bb8309e2c51348f8', 'request_id': 'req-0ea9486e-2bc7-4a50-9a17-3ed74f410826'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.908200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:52.913045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38b6d7b146a34896bb8309e2c51348f8', 'request_id': 'req-0ea9486e-2bc7-4a50-9a17-3ed74f410826'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.919256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38b6d7b146a34896bb8309e2c51348f8', 'request_id': 'req-0ea9486e-2bc7-4a50-9a17-3ed74f410826'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.934068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] The token cache is being invalidate because project 38b6d7b146a34896bb8309e2c51348f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:06:52.934403 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.935034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ea9486e-2bc7-4a50-9a17-3ed74f410826 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:06:52.936714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1242/2520] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:06:52 2022] DELETE /identity/v3/projects/38b6d7b146a34896bb8309e2c51348f8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:06:52.941881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8d66b-d504-4369-8ab0-591557090e10 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:52.942198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8d66b-d504-4369-8ab0-591557090e10 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:52.942473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08d8d66b-d504-4369-8ab0-591557090e10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:52.963561 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-08d8d66b-d504-4369-8ab0-591557090e10 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:06:52.964779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-08d8d66b-d504-4369-8ab0-591557090e10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:53.028392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1279/2521] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:06:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:06:53.036685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11cec6-ca65-4e46-ab2d-3d250cff0a6d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:53.036973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11cec6-ca65-4e46-ab2d-3d250cff0a6d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:53.037410 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b11cec6-ca65-4e46-ab2d-3d250cff0a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:53.058843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-6b11cec6-ca65-4e46-ab2d-3d250cff0a6d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:06:53.060098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6b11cec6-ca65-4e46-ab2d-3d250cff0a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:53.069366 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1243/2522] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:06:53.073818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8d66b-d504-4369-8ab0-591557090e10 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:53.084060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08d8d66b-d504-4369-8ab0-591557090e10 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:53.089334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c9640-5609-4927-9584-fa54d2aa8f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:53.089951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c9640-5609-4927-9584-fa54d2aa8f88 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:53.090264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c9640-5609-4927-9584-fa54d2aa8f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:53.090551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c9640-5609-4927-9584-fa54d2aa8f88 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:53.091197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c9640-5609-4927-9584-fa54d2aa8f88 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:06:53.091499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c9640-5609-4927-9584-fa54d2aa8f88 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:53.093108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836c9640-5609-4927-9584-fa54d2aa8f88 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:53.100247 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:53.106011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1280/2523] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:06:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:06:53.110459 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11cec6-ca65-4e46-ab2d-3d250cff0a6d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:53.121012 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b11cec6-ca65-4e46-ab2d-3d250cff0a6d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:53.125849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe08b7-ddba-45c1-8357-fe32415ad6d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:53.126481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe08b7-ddba-45c1-8357-fe32415ad6d1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:53.126760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe08b7-ddba-45c1-8357-fe32415ad6d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:53.127031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fe08b7-ddba-45c1-8357-fe32415ad6d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:53.127656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe08b7-ddba-45c1-8357-fe32415ad6d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:53.129554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fe08b7-ddba-45c1-8357-fe32415ad6d1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:53.160402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1244/2524] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:06:53 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:06:53.166872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c9640-5609-4927-9584-fa54d2aa8f88 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:53.177963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836c9640-5609-4927-9584-fa54d2aa8f88 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:53.237648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792fe94-eeeb-4bc0-ae15-4c5bf7013ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:53.238286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792fe94-eeeb-4bc0-ae15-4c5bf7013ba8 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:53.238564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792fe94-eeeb-4bc0-ae15-4c5bf7013ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:53.238839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792fe94-eeeb-4bc0-ae15-4c5bf7013ba8 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:53.239469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792fe94-eeeb-4bc0-ae15-4c5bf7013ba8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:53.241402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f792fe94-eeeb-4bc0-ae15-4c5bf7013ba8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:53.263674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1281/2525] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:06:53 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:06:53.267835 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe08b7-ddba-45c1-8357-fe32415ad6d1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:53.278234 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fe08b7-ddba-45c1-8357-fe32415ad6d1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:53.283331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4538d5-0d8a-40be-a4b8-806ce30d5893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:53.283875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4538d5-0d8a-40be-a4b8-806ce30d5893 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:53.284153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4538d5-0d8a-40be-a4b8-806ce30d5893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:53.284421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4538d5-0d8a-40be-a4b8-806ce30d5893 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:53.285060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4538d5-0d8a-40be-a4b8-806ce30d5893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:53.286737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4538d5-0d8a-40be-a4b8-806ce30d5893 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:53.296841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1245/2526] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:06:53.301039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792fe94-eeeb-4bc0-ae15-4c5bf7013ba8 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:53.311587 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792fe94-eeeb-4bc0-ae15-4c5bf7013ba8 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:53.316397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36f54ad-3f34-49d5-82a1-0fb8304a2295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:53.316992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f54ad-3f34-49d5-82a1-0fb8304a2295 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:53.317272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f54ad-3f34-49d5-82a1-0fb8304a2295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:53.317529 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36f54ad-3f34-49d5-82a1-0fb8304a2295 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:53.318299 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f54ad-3f34-49d5-82a1-0fb8304a2295 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:53.319885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36f54ad-3f34-49d5-82a1-0fb8304a2295 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:53.330533 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1282/2527] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:06:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:06:53.334841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4538d5-0d8a-40be-a4b8-806ce30d5893 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:53.345396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4538d5-0d8a-40be-a4b8-806ce30d5893 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:53.350531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3f304-7eda-4fff-916f-225538bd4dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:53.351192 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3f304-7eda-4fff-916f-225538bd4dad admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:53.351472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3f304-7eda-4fff-916f-225538bd4dad admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:53.351746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a3f304-7eda-4fff-916f-225538bd4dad admin admin] PATH_INFO: `/v3/projects/0f2d19ffb32443babaa154c5766b6982/users/a8576530b4ed4749a1e138f975d9b8b9/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:53.367092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3f304-7eda-4fff-916f-225538bd4dad admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f2d19ffb32443babaa154c5766b6982, user_id=a8576530b4ed4749a1e138f975d9b8b9, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:53.369739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a3f304-7eda-4fff-916f-225538bd4dad admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:53.384861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1246/2528] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:06:53 2022] PUT /identity/v3/projects/0f2d19ffb32443babaa154c5766b6982/users/a8576530b4ed4749a1e138f975d9b8b9/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:06:53.390241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b37f39-d672-4936-be0a-aee422ffd36e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:53.390531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b37f39-d672-4936-be0a-aee422ffd36e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:53.390810 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b37f39-d672-4936-be0a-aee422ffd36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:53.410098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-45b37f39-d672-4936-be0a-aee422ffd36e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:06:53.411439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-45b37f39-d672-4936-be0a-aee422ffd36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:53.473941 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1283/2529] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:06:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:06:53.484021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3f304-7eda-4fff-916f-225538bd4dad admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:53.486339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-19a3f304-7eda-4fff-916f-225538bd4dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:53.550484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a3f304-7eda-4fff-916f-225538bd4dad admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:53.554988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d6afe-7413-4811-8d4d-814a6b246df0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:53.555933 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6afe-7413-4811-8d4d-814a6b246df0 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:53.556206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6afe-7413-4811-8d4d-814a6b246df0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:53.556475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d6afe-7413-4811-8d4d-814a6b246df0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:53.566644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d6afe-7413-4811-8d4d-814a6b246df0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:53.568082 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d6afe-7413-4811-8d4d-814a6b246df0 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:53.577760 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1247/2530] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:57.671864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9556f-72ae-47de-9e31-0c8272fb1700 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:57.672493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9556f-72ae-47de-9e31-0c8272fb1700 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:57.672493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b9556f-72ae-47de-9e31-0c8272fb1700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:57.701923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-05b9556f-72ae-47de-9e31-0c8272fb1700 None None] MFA Rules not processed for user `a8576530b4ed4749a1e138f975d9b8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:06:57.703187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-05b9556f-72ae-47de-9e31-0c8272fb1700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:57.784035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1284/2531] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:06:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:06:57.791584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d6afe-7413-4811-8d4d-814a6b246df0 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:57.793468 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9a6d6afe-7413-4811-8d4d-814a6b246df0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:57.927311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d6afe-7413-4811-8d4d-814a6b246df0 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:57.934475 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9709ab3f-98c6-42f3-b5a6-c8965cc14c79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:57.935451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709ab3f-98c6-42f3-b5a6-c8965cc14c79 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:57.935920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709ab3f-98c6-42f3-b5a6-c8965cc14c79 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:57.936374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9709ab3f-98c6-42f3-b5a6-c8965cc14c79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:57.947256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709ab3f-98c6-42f3-b5a6-c8965cc14c79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:57.949559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9709ab3f-98c6-42f3-b5a6-c8965cc14c79 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:57.965247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1248/2532] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:06:59.402478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59df49e-cc42-4643-addf-b808adac8901 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:59.403043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59df49e-cc42-4643-addf-b808adac8901 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:59.403043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59df49e-cc42-4643-addf-b808adac8901 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:59.417435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:59.424058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a59df49e-cc42-4643-addf-b808adac8901 None None] MFA Rules not processed for user `a8576530b4ed4749a1e138f975d9b8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:06:59.425939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a59df49e-cc42-4643-addf-b808adac8901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:59.433426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1285/2533] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:06:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:06:59.559414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a41494-aea5-4cc3-bd31-442a22a517b7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:59.559950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a41494-aea5-4cc3-bd31-442a22a517b7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:59.560178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a41494-aea5-4cc3-bd31-442a22a517b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:59.568265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:06:59.581348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-85a41494-aea5-4cc3-bd31-442a22a517b7 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:06:59.583382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-85a41494-aea5-4cc3-bd31-442a22a517b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:59.660180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1249/2534] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:06:59 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:06:59.673930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59df49e-cc42-4643-addf-b808adac8901 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:59.676418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a59df49e-cc42-4643-addf-b808adac8901 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:59.781509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a41494-aea5-4cc3-bd31-442a22a517b7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:59.783247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-85a41494-aea5-4cc3-bd31-442a22a517b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:59.784641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59df49e-cc42-4643-addf-b808adac8901 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:59.790239 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d1834-1291-4f68-a732-04dcef366b51 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:59.791169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d1834-1291-4f68-a732-04dcef366b51 service swift] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:59.791677 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d1834-1291-4f68-a732-04dcef366b51 service swift] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:59.791996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d1834-1291-4f68-a732-04dcef366b51 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:59.804481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d1834-1291-4f68-a732-04dcef366b51 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:59.806773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d1834-1291-4f68-a732-04dcef366b51 service swift] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:59.809294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a41494-aea5-4cc3-bd31-442a22a517b7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:59.815431 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642c08a3-f94b-42c8-a6aa-7aaf2485d9b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:59.816166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c08a3-f94b-42c8-a6aa-7aaf2485d9b8 service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:59.816453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c08a3-f94b-42c8-a6aa-7aaf2485d9b8 service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:59.816730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642c08a3-f94b-42c8-a6aa-7aaf2485d9b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:59.816993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1286/2535] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:06:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:06:59.817815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d1834-1291-4f68-a732-04dcef366b51 service swift] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:06:59.818786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-642c08a3-f94b-42c8-a6aa-7aaf2485d9b8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:59.819509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a07d1834-1291-4f68-a732-04dcef366b51 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:59.833788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c08a3-f94b-42c8-a6aa-7aaf2485d9b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:59.836375 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642c08a3-f94b-42c8-a6aa-7aaf2485d9b8 service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:59.837739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d1834-1291-4f68-a732-04dcef366b51 service swift] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:06:59.842228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83abe7db-48f0-427c-9c55-ed2e3bfcb8e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:06:59.843172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abe7db-48f0-427c-9c55-ed2e3bfcb8e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:06:59.843535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abe7db-48f0-427c-9c55-ed2e3bfcb8e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:06:59.844076 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83abe7db-48f0-427c-9c55-ed2e3bfcb8e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:06:59.848013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-83abe7db-48f0-427c-9c55-ed2e3bfcb8e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:06:59.851172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1250/2536] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:06:59.884144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83abe7db-48f0-427c-9c55-ed2e3bfcb8e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:06:59.885676 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83abe7db-48f0-427c-9c55-ed2e3bfcb8e1 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:06:59.896379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1287/2537] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:06:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:09.845127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=29 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:09.890821 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=38 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:15.425844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045f16e-f556-4e91-8536-c3e08e1ecfcd None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:15.426500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045f16e-f556-4e91-8536-c3e08e1ecfcd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:15.426500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045f16e-f556-4e91-8536-c3e08e1ecfcd None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:15.427803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1251/2538] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:07:15.434586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4fc56-bb9b-42c6-b2fd-276d1bf211b4 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:15.435077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4fc56-bb9b-42c6-b2fd-276d1bf211b4 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:15.435195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a4fc56-bb9b-42c6-b2fd-276d1bf211b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:15.456764 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d1a4fc56-bb9b-42c6-b2fd-276d1bf211b4 None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:15.458000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d1a4fc56-bb9b-42c6-b2fd-276d1bf211b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:15.469858 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1288/2539] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:07:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:15.482453 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045f16e-f556-4e91-8536-c3e08e1ecfcd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:15.484145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2045f16e-f556-4e91-8536-c3e08e1ecfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:15.545500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045f16e-f556-4e91-8536-c3e08e1ecfcd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:15.549178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1de2197-c0f8-4583-9b55-3876a077d43f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3505c8965c4a42e493a2f5916cab56dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:15.549800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de2197-c0f8-4583-9b55-3876a077d43f service glance] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:15.550089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de2197-c0f8-4583-9b55-3876a077d43f service glance] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:15.550379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1de2197-c0f8-4583-9b55-3876a077d43f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:15.561787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de2197-c0f8-4583-9b55-3876a077d43f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:15.563271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1de2197-c0f8-4583-9b55-3876a077d43f service glance] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:15.574374 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1252/2540] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:15.654001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead65ca-79b4-42b8-9ad8-3b44cba835e6 None None] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:15.654236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead65ca-79b4-42b8-9ad8-3b44cba835e6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:15.654568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ead65ca-79b4-42b8-9ad8-3b44cba835e6 None None] PATH_INFO: `/` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:15.656441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1289/2541] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:07:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:07:15.662614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ee756-5a91-40c1-b0d8-95da276ef9ec None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:15.662889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ee756-5a91-40c1-b0d8-95da276ef9ec None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:15.663179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7ee756-5a91-40c1-b0d8-95da276ef9ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:15.685921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ea7ee756-5a91-40c1-b0d8-95da276ef9ec None None] MFA Rules not processed for user `f94aa8af227a42a08d8a0e4b7f02b85a`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:15.687175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ea7ee756-5a91-40c1-b0d8-95da276ef9ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:15.694705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1253/2542] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:07:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:15.707205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead65ca-79b4-42b8-9ad8-3b44cba835e6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:15.719034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ead65ca-79b4-42b8-9ad8-3b44cba835e6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:15.723166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fb76ef-0985-402e-88e4-2801a25495b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:15.723775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb76ef-0985-402e-88e4-2801a25495b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:15.724054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb76ef-0985-402e-88e4-2801a25495b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:15.724310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fb76ef-0985-402e-88e4-2801a25495b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:15.731414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb76ef-0985-402e-88e4-2801a25495b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:15.732852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fb76ef-0985-402e-88e4-2801a25495b1 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:15.740494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1290/2543] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:15.824091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad087204-6ba4-4af7-8c7a-6cb3ba827bc2 None None] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:15.824380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad087204-6ba4-4af7-8c7a-6cb3ba827bc2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:15.824697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad087204-6ba4-4af7-8c7a-6cb3ba827bc2 None None] PATH_INFO: `/` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:15.826070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1254/2544] 10.222.0.25 () {54 vars in 921 bytes} [Tue Sep 13 07:07:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 13 07:07:15.831749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0eb40-50d3-4a86-9694-e69c6992981f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:15.832033 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0eb40-50d3-4a86-9694-e69c6992981f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:15.832300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad0eb40-50d3-4a86-9694-e69c6992981f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:15.852800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6ad0eb40-50d3-4a86-9694-e69c6992981f None None] MFA Rules not processed for user `87f3a9ffb88f4faa85edc3dabce0c8f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:15.853997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6ad0eb40-50d3-4a86-9694-e69c6992981f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:15.863495 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1291/2545] 10.222.0.25 () {58 vars in 1018 bytes} [Tue Sep 13 07:07:15 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:15.874036 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad087204-6ba4-4af7-8c7a-6cb3ba827bc2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:15.886227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad087204-6ba4-4af7-8c7a-6cb3ba827bc2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:15.890128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41753b8c-8af4-4680-b201-aae952e6a2fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:15.890747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753b8c-8af4-4680-b201-aae952e6a2fd service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:15.891032 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753b8c-8af4-4680-b201-aae952e6a2fd service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:15.891304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41753b8c-8af4-4680-b201-aae952e6a2fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:15.901742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41753b8c-8af4-4680-b201-aae952e6a2fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:15.903233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41753b8c-8af4-4680-b201-aae952e6a2fd service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:15.914147 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1255/2546] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:19.380038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027c8fd-17b8-4ba8-a780-466e6450b5ff None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:19.380736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027c8fd-17b8-4ba8-a780-466e6450b5ff None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:19.380736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d027c8fd-17b8-4ba8-a780-466e6450b5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:19.402063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d027c8fd-17b8-4ba8-a780-466e6450b5ff None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:19.403384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d027c8fd-17b8-4ba8-a780-466e6450b5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:19.411103 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1292/2547] 10.222.0.25 () {58 vars in 1009 bytes} [Tue Sep 13 07:07:19 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:19.424733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41753b8c-8af4-4680-b201-aae952e6a2fd service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:19.448146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41753b8c-8af4-4680-b201-aae952e6a2fd service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:19.459599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d883da6-c373-41ff-a171-cb9caf9d625c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:19.460278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d883da6-c373-41ff-a171-cb9caf9d625c service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:19.460658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d883da6-c373-41ff-a171-cb9caf9d625c service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:19.461141 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d883da6-c373-41ff-a171-cb9caf9d625c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:19.471523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d883da6-c373-41ff-a171-cb9caf9d625c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:19.473271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d883da6-c373-41ff-a171-cb9caf9d625c service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:19.494319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1256/2548] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:07:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 13 07:07:29.395172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=15 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:29.478704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=41 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:34.438388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027c8fd-17b8-4ba8-a780-466e6450b5ff None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:34.465008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d027c8fd-17b8-4ba8-a780-466e6450b5ff None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:34.471950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a0d115-9037-4915-a425-5e82554be088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:34.472907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0d115-9037-4915-a425-5e82554be088 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:34.473189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0d115-9037-4915-a425-5e82554be088 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:34.473457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a0d115-9037-4915-a425-5e82554be088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:34.479240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f5a0d115-9037-4915-a425-5e82554be088 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:34.500775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0d115-9037-4915-a425-5e82554be088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:34.504119 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a0d115-9037-4915-a425-5e82554be088 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:34.534489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1293/2549] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:34 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:35.892319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0ef15-4076-478a-92b6-9734eeac3316 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:35.892940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0ef15-4076-478a-92b6-9734eeac3316 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:35.892940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0ef15-4076-478a-92b6-9734eeac3316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:35.918818 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d7b0ef15-4076-478a-92b6-9734eeac3316 None None] MFA Rules not processed for user `67fa873d16414749bc478a59de3ec5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:35.920074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d7b0ef15-4076-478a-92b6-9734eeac3316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:35.979859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1257/2550] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:37.444814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54354887-29cc-432c-a1a5-dae5448eea13 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:37.445469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54354887-29cc-432c-a1a5-dae5448eea13 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:37.446263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54354887-29cc-432c-a1a5-dae5448eea13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:37.467921 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-54354887-29cc-432c-a1a5-dae5448eea13 None None] MFA Rules not processed for user `7363637f44264a65b66ba53da825eaff`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:37.469151 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-54354887-29cc-432c-a1a5-dae5448eea13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:37.477215 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1294/2551] 10.222.0.25 () {58 vars in 967 bytes} [Tue Sep 13 07:07:37 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:37.499332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0ef15-4076-478a-92b6-9734eeac3316 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:37.511465 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0ef15-4076-478a-92b6-9734eeac3316 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:37.517399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2acf9e-5cd0-4255-9112-06e520033e88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4815ef133a3a4604bc467c222d450fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:37.518013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2acf9e-5cd0-4255-9112-06e520033e88 service swift] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:37.518334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2acf9e-5cd0-4255-9112-06e520033e88 service swift] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:37.518602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2acf9e-5cd0-4255-9112-06e520033e88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:37.527684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2acf9e-5cd0-4255-9112-06e520033e88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:37.529282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2acf9e-5cd0-4255-9112-06e520033e88 service swift] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:37.537600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1258/2552] 10.222.0.25 () {60 vars in 1384 bytes} [Tue Sep 13 07:07:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 13 07:07:39.423372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54354887-29cc-432c-a1a5-dae5448eea13 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:39.437392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54354887-29cc-432c-a1a5-dae5448eea13 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:39.460336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e583a5d-b7ca-4a8c-b6ae-daa83a81ef07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:39.461224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e583a5d-b7ca-4a8c-b6ae-daa83a81ef07 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:39.461722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e583a5d-b7ca-4a8c-b6ae-daa83a81ef07 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:39.462538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e583a5d-b7ca-4a8c-b6ae-daa83a81ef07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:39.464780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9e583a5d-b7ca-4a8c-b6ae-daa83a81ef07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:39.480148 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e583a5d-b7ca-4a8c-b6ae-daa83a81ef07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:39.481621 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e583a5d-b7ca-4a8c-b6ae-daa83a81ef07 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:39.492619 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1295/2553] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:42.839396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2acf9e-5cd0-4255-9112-06e520033e88 service swift] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:42.841075 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ba2acf9e-5cd0-4255-9112-06e520033e88 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:42.856872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2acf9e-5cd0-4255-9112-06e520033e88 service swift] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:42.861394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:42.862038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:42.862353 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:42.862624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] PATH_INFO: `/v3/users/5a0fd7347cad41bc80f427f872cf78fa` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:42.863742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a0fd7347cad41bc80f427f872cf78fa)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:42.865676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:42.919559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a0fd7347cad41bc80f427f872cf78fa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:42.928932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a0fd7347cad41bc80f427f872cf78fa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:42.943164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a0fd7347cad41bc80f427f872cf78fa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:42.943975 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a0fd7347cad41bc80f427f872cf78fa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:42.952344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c0811530-6752-4616-a117-c81dcd5fe182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a0fd7347cad41bc80f427f872cf78fa'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:42.959188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1259/2554] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:07:42 2022] DELETE /identity/v3/users/5a0fd7347cad41bc80f427f872cf78fa => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:43.173047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e583a5d-b7ca-4a8c-b6ae-daa83a81ef07 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:43.183888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e583a5d-b7ca-4a8c-b6ae-daa83a81ef07 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:43.243791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:43.244448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.244731 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.245006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] PATH_INFO: `/v3/projects/8c7dc05c1c9d40b2bb0defc9dd157d6e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.246179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c7dc05c1c9d40b2bb0defc9dd157d6e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:43.248088 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:43.295931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c7dc05c1c9d40b2bb0defc9dd157d6e', 'request_id': 'req-b2799532-8fac-4417-a388-76201e5695bc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:43.301894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c7dc05c1c9d40b2bb0defc9dd157d6e', 'request_id': 'req-b2799532-8fac-4417-a388-76201e5695bc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:43.308768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c7dc05c1c9d40b2bb0defc9dd157d6e', 'request_id': 'req-b2799532-8fac-4417-a388-76201e5695bc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:43.323697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] The token cache is being invalidate because project 8c7dc05c1c9d40b2bb0defc9dd157d6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:07:43.324000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:43.324625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b2799532-8fac-4417-a388-76201e5695bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:43.326237 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1296/2555] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:07:43 2022] DELETE /identity/v3/projects/8c7dc05c1c9d40b2bb0defc9dd157d6e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:43.331434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15802c20-e14f-47ff-8350-590c9d94a5ae None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.331717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15802c20-e14f-47ff-8350-590c9d94a5ae None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.331985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15802c20-e14f-47ff-8350-590c9d94a5ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.352303 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-15802c20-e14f-47ff-8350-590c9d94a5ae None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:43.353646 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-15802c20-e14f-47ff-8350-590c9d94a5ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:43.417767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1260/2556] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:43.427090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cfe49-4b00-4e66-b038-b789247c1c5f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.427374 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cfe49-4b00-4e66-b038-b789247c1c5f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.427644 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f74cfe49-4b00-4e66-b038-b789247c1c5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.449161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f74cfe49-4b00-4e66-b038-b789247c1c5f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:43.450449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f74cfe49-4b00-4e66-b038-b789247c1c5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:43.459581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1297/2557] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:43.464891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15802c20-e14f-47ff-8350-590c9d94a5ae None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:43.471694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:43.476435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15802c20-e14f-47ff-8350-590c9d94a5ae None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:43.483409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9e69-8fbf-49e6-b791-42a03ab60148 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:43.484152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9e69-8fbf-49e6-b791-42a03ab60148 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.484509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9e69-8fbf-49e6-b791-42a03ab60148 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.484864 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fec9e69-8fbf-49e6-b791-42a03ab60148 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.485597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9e69-8fbf-49e6-b791-42a03ab60148 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:07:43.486218 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9e69-8fbf-49e6-b791-42a03ab60148 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:43.489017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fec9e69-8fbf-49e6-b791-42a03ab60148 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:43.501692 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1261/2558] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:07:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:07:43.506490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cfe49-4b00-4e66-b038-b789247c1c5f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:43.516703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f74cfe49-4b00-4e66-b038-b789247c1c5f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:43.521835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1df3f-df9d-4a78-917f-60835833791b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:43.522481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1df3f-df9d-4a78-917f-60835833791b admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.522756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1df3f-df9d-4a78-917f-60835833791b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.523027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a1df3f-df9d-4a78-917f-60835833791b admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.523667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1df3f-df9d-4a78-917f-60835833791b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:43.525604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a1df3f-df9d-4a78-917f-60835833791b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:43.556030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1298/2559] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:07:43 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:43.560464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9e69-8fbf-49e6-b791-42a03ab60148 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:43.570996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fec9e69-8fbf-49e6-b791-42a03ab60148 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:43.639971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3406ff9-ac02-4934-9c00-4f350ddc1675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:43.640597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3406ff9-ac02-4934-9c00-4f350ddc1675 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.640878 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3406ff9-ac02-4934-9c00-4f350ddc1675 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.641160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3406ff9-ac02-4934-9c00-4f350ddc1675 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.641791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3406ff9-ac02-4934-9c00-4f350ddc1675 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:43.643773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3406ff9-ac02-4934-9c00-4f350ddc1675 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:43.667400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1262/2560] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:07:43 2022] POST /identity/v3/users => generated 303 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:43.671727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1df3f-df9d-4a78-917f-60835833791b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:43.682905 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a1df3f-df9d-4a78-917f-60835833791b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:43.687665 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5507d53-2a24-4c29-b671-8bc5ef9ea7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:43.688270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5507d53-2a24-4c29-b671-8bc5ef9ea7a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.688546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5507d53-2a24-4c29-b671-8bc5ef9ea7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.688823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5507d53-2a24-4c29-b671-8bc5ef9ea7a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.689457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5507d53-2a24-4c29-b671-8bc5ef9ea7a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:43.691122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5507d53-2a24-4c29-b671-8bc5ef9ea7a9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:43.701180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1299/2561] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:43.704956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3406ff9-ac02-4934-9c00-4f350ddc1675 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:43.715085 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3406ff9-ac02-4934-9c00-4f350ddc1675 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:43.719919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ce9030-5bf8-4d21-b0d5-1015f259a2b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:43.720527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce9030-5bf8-4d21-b0d5-1015f259a2b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.720813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce9030-5bf8-4d21-b0d5-1015f259a2b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.721072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ce9030-5bf8-4d21-b0d5-1015f259a2b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.721718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce9030-5bf8-4d21-b0d5-1015f259a2b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:43.723368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ce9030-5bf8-4d21-b0d5-1015f259a2b5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:43.734263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1263/2562] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:07:43.738405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5507d53-2a24-4c29-b671-8bc5ef9ea7a9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:43.750353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5507d53-2a24-4c29-b671-8bc5ef9ea7a9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:43.759669 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d14aad-eacd-4e25-9f52-7ff3323def28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:43.760626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d14aad-eacd-4e25-9f52-7ff3323def28 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.761037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d14aad-eacd-4e25-9f52-7ff3323def28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.761386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d14aad-eacd-4e25-9f52-7ff3323def28 admin admin] PATH_INFO: `/v3/projects/23b4935eb2f949b3a1e845bcaa4e275b/users/df27059828ef482ea7e33d8941050be7/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.780064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d14aad-eacd-4e25-9f52-7ff3323def28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23b4935eb2f949b3a1e845bcaa4e275b, user_id=df27059828ef482ea7e33d8941050be7, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:43.782777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d14aad-eacd-4e25-9f52-7ff3323def28 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:43.784971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:43.799908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1300/2563] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:43 2022] PUT /identity/v3/projects/23b4935eb2f949b3a1e845bcaa4e275b/users/df27059828ef482ea7e33d8941050be7/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:43.805895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606054f5-f87c-4a15-85fc-6bda807d488d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.806452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606054f5-f87c-4a15-85fc-6bda807d488d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.806726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606054f5-f87c-4a15-85fc-6bda807d488d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.826269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-606054f5-f87c-4a15-85fc-6bda807d488d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:43.827484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-606054f5-f87c-4a15-85fc-6bda807d488d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:43.894767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1264/2564] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:43.905625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d14aad-eacd-4e25-9f52-7ff3323def28 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:43.907428 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-34d14aad-eacd-4e25-9f52-7ff3323def28 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:43.968747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d14aad-eacd-4e25-9f52-7ff3323def28 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:43.973129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac145a-4d0d-4829-8f68-0f1f7785e876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:43.973746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac145a-4d0d-4829-8f68-0f1f7785e876 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:43.974026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac145a-4d0d-4829-8f68-0f1f7785e876 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:43.974352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0ac145a-4d0d-4829-8f68-0f1f7785e876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:43.984887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac145a-4d0d-4829-8f68-0f1f7785e876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:43.986401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0ac145a-4d0d-4829-8f68-0f1f7785e876 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:43.996948 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1301/2565] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:44.749624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a583a-9077-48a2-9cf2-a7820c169e41 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:44.750128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a583a-9077-48a2-9cf2-a7820c169e41 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:44.750560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433a583a-9077-48a2-9cf2-a7820c169e41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:44.781008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-433a583a-9077-48a2-9cf2-a7820c169e41 None None] MFA Rules not processed for user `df27059828ef482ea7e33d8941050be7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:44.783682 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-433a583a-9077-48a2-9cf2-a7820c169e41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:44.872461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:44.879308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1265/2566] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:44 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:44.883210 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac145a-4d0d-4829-8f68-0f1f7785e876 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:44.893451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0ac145a-4d0d-4829-8f68-0f1f7785e876 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:44.899027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52909552-9842-4069-9489-78180afeaaaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:44.899660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52909552-9842-4069-9489-78180afeaaaf admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:44.900009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52909552-9842-4069-9489-78180afeaaaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:44.900434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52909552-9842-4069-9489-78180afeaaaf admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:44.901069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52909552-9842-4069-9489-78180afeaaaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:44.903070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52909552-9842-4069-9489-78180afeaaaf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:44.933270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1302/2567] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:07:44 2022] POST /identity/v3/projects => generated 405 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:44.937604 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a583a-9077-48a2-9cf2-a7820c169e41 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:44.950372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433a583a-9077-48a2-9cf2-a7820c169e41 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:45.008537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb6a8b-bb2b-4124-b845-ba0e395b28e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:45.009197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb6a8b-bb2b-4124-b845-ba0e395b28e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:45.009464 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb6a8b-bb2b-4124-b845-ba0e395b28e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:45.009733 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cb6a8b-bb2b-4124-b845-ba0e395b28e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:45.010397 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb6a8b-bb2b-4124-b845-ba0e395b28e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:45.012333 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cb6a8b-bb2b-4124-b845-ba0e395b28e8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:45.035132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1266/2568] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:07:44 2022] POST /identity/v3/users => generated 309 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:45.039876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52909552-9842-4069-9489-78180afeaaaf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:45.051132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52909552-9842-4069-9489-78180afeaaaf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:45.057140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a790cdd-6988-4390-82cc-f957801dce1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:45.057763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a790cdd-6988-4390-82cc-f957801dce1a admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:45.058048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a790cdd-6988-4390-82cc-f957801dce1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:45.058346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a790cdd-6988-4390-82cc-f957801dce1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:45.058993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a790cdd-6988-4390-82cc-f957801dce1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:45.060696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a790cdd-6988-4390-82cc-f957801dce1a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:45.072376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1303/2569] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:45.076745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb6a8b-bb2b-4124-b845-ba0e395b28e8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:45.087068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cb6a8b-bb2b-4124-b845-ba0e395b28e8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:45.092023 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca08c0-df37-4783-9157-025f0d6b3b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:45.092677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca08c0-df37-4783-9157-025f0d6b3b0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:45.092954 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca08c0-df37-4783-9157-025f0d6b3b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:45.093220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddca08c0-df37-4783-9157-025f0d6b3b0a admin admin] PATH_INFO: `/v3/projects/25223fe8ff4a4f41b98b5d0aafcd92a9/users/65bac14264f74ee3889a3823c58eb071/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:45.110491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca08c0-df37-4783-9157-025f0d6b3b0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=25223fe8ff4a4f41b98b5d0aafcd92a9, user_id=65bac14264f74ee3889a3823c58eb071, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:45.113169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddca08c0-df37-4783-9157-025f0d6b3b0a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:45.131958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1267/2570] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:45 2022] PUT /identity/v3/projects/25223fe8ff4a4f41b98b5d0aafcd92a9/users/65bac14264f74ee3889a3823c58eb071/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:45.136788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a790cdd-6988-4390-82cc-f957801dce1a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:45.147412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a790cdd-6988-4390-82cc-f957801dce1a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:45.208105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ffd31b-4c58-4236-8a2d-8cc7252e2728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:45.208660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffd31b-4c58-4236-8a2d-8cc7252e2728 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:45.208965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffd31b-4c58-4236-8a2d-8cc7252e2728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:45.209225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ffd31b-4c58-4236-8a2d-8cc7252e2728 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:45.209858 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffd31b-4c58-4236-8a2d-8cc7252e2728 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:45.211591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ffd31b-4c58-4236-8a2d-8cc7252e2728 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:45.222435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1304/2571] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:45.226677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca08c0-df37-4783-9157-025f0d6b3b0a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:45.237667 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddca08c0-df37-4783-9157-025f0d6b3b0a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:45.242535 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91af2-8802-444d-b855-4f1787dc02f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:45.243255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91af2-8802-444d-b855-4f1787dc02f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:45.243545 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91af2-8802-444d-b855-4f1787dc02f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:45.243791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b91af2-8802-444d-b855-4f1787dc02f9 admin admin] PATH_INFO: `/v3/projects/25223fe8ff4a4f41b98b5d0aafcd92a9/users/65bac14264f74ee3889a3823c58eb071/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:45.245661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b91af2-8802-444d-b855-4f1787dc02f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25223fe8ff4a4f41b98b5d0aafcd92a9, user_id=65bac14264f74ee3889a3823c58eb071, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:45.248310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b91af2-8802-444d-b855-4f1787dc02f9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:45.258395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1268/2572] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:45 2022] PUT /identity/v3/projects/25223fe8ff4a4f41b98b5d0aafcd92a9/users/65bac14264f74ee3889a3823c58eb071/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:45.951356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e568a75-bca6-4539-ba42-8d0a761fdb6f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:45.951958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e568a75-bca6-4539-ba42-8d0a761fdb6f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:45.951958 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e568a75-bca6-4539-ba42-8d0a761fdb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:45.987095 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2e568a75-bca6-4539-ba42-8d0a761fdb6f None None] MFA Rules not processed for user `65bac14264f74ee3889a3823c58eb071`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:45.988477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2e568a75-bca6-4539-ba42-8d0a761fdb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:46.051514 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:46.088349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1305/2573] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:45 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:47.211030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91af2-8802-444d-b855-4f1787dc02f9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:47.212838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f7b91af2-8802-444d-b855-4f1787dc02f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:47.296641 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b91af2-8802-444d-b855-4f1787dc02f9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:47.297480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e568a75-bca6-4539-ba42-8d0a761fdb6f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:47.302314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa23f36-56a7-4df9-bd7e-af7bff854add service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:47.303045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa23f36-56a7-4df9-bd7e-af7bff854add service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:47.303422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa23f36-56a7-4df9-bd7e-af7bff854add service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:47.303894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa23f36-56a7-4df9-bd7e-af7bff854add service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:47.306484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6fa23f36-56a7-4df9-bd7e-af7bff854add service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:47.330399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa23f36-56a7-4df9-bd7e-af7bff854add service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:47.330399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa23f36-56a7-4df9-bd7e-af7bff854add service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:47.336796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1269/2574] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:47.372476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e568a75-bca6-4539-ba42-8d0a761fdb6f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:47.376924 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:47.377560 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:47.377837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:47.378135 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] PATH_INFO: `/v3/users/df27059828ef482ea7e33d8941050be7` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:47.379234 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df27059828ef482ea7e33d8941050be7)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:47.381166 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:47.439745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df27059828ef482ea7e33d8941050be7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.440408 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df27059828ef482ea7e33d8941050be7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.449038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df27059828ef482ea7e33d8941050be7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.461037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df27059828ef482ea7e33d8941050be7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.461818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df27059828ef482ea7e33d8941050be7'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.474802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1306/2575] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:07:47 2022] DELETE /identity/v3/users/df27059828ef482ea7e33d8941050be7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:47.653039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa23f36-56a7-4df9-bd7e-af7bff854add service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:47.664188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa23f36-56a7-4df9-bd7e-af7bff854add service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:47.709076 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:47.727893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:47.728565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:47.728861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:47.729134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] PATH_INFO: `/v3/projects/23b4935eb2f949b3a1e845bcaa4e275b` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:47.730307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23b4935eb2f949b3a1e845bcaa4e275b)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:47.732242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:47.784540 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23b4935eb2f949b3a1e845bcaa4e275b', 'request_id': 'req-5e957540-745d-44f8-87e2-9a342dabe5d8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.791766 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23b4935eb2f949b3a1e845bcaa4e275b', 'request_id': 'req-5e957540-745d-44f8-87e2-9a342dabe5d8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.798672 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23b4935eb2f949b3a1e845bcaa4e275b', 'request_id': 'req-5e957540-745d-44f8-87e2-9a342dabe5d8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.814599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] The token cache is being invalidate because project 23b4935eb2f949b3a1e845bcaa4e275b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:07:47.814924 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.815596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.817021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1270/2576] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:07:47 2022] DELETE /identity/v3/projects/23b4935eb2f949b3a1e845bcaa4e275b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:47.822746 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:47.824176 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:47.838025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fbac710-7f3e-4841-b976-560ef785aff3 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:47.897588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:47.898168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:47.898448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:47.898756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] PATH_INFO: `/v3/users/65bac14264f74ee3889a3823c58eb071` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:47.900470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65bac14264f74ee3889a3823c58eb071)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:47.902466 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:47.907170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:47.958218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65bac14264f74ee3889a3823c58eb071'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.958871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65bac14264f74ee3889a3823c58eb071'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.967986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65bac14264f74ee3889a3823c58eb071'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.980554 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65bac14264f74ee3889a3823c58eb071'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.981225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5ce2a150-d997-4171-87f9-35066c059a46 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65bac14264f74ee3889a3823c58eb071'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:47.994599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1307/2577] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:07:47 2022] DELETE /identity/v3/users/65bac14264f74ee3889a3823c58eb071 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:48.170776 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:48.172364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:48.186040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e957540-745d-44f8-87e2-9a342dabe5d8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:48.249858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:48.250634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.250803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.251080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] PATH_INFO: `/v3/projects/25223fe8ff4a4f41b98b5d0aafcd92a9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.252230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] RBAC: Authorizing `identity:delete_project(project_id=25223fe8ff4a4f41b98b5d0aafcd92a9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:48.254203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:48.308533 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25223fe8ff4a4f41b98b5d0aafcd92a9', 'request_id': 'req-872a4a76-7259-435c-a6ed-c2ed9174a4cd'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:48.314573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25223fe8ff4a4f41b98b5d0aafcd92a9', 'request_id': 'req-872a4a76-7259-435c-a6ed-c2ed9174a4cd'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:48.320145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25223fe8ff4a4f41b98b5d0aafcd92a9', 'request_id': 'req-872a4a76-7259-435c-a6ed-c2ed9174a4cd'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:48.333662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] The token cache is being invalidate because project 25223fe8ff4a4f41b98b5d0aafcd92a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:07:48.333956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:48.338501 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-872a4a76-7259-435c-a6ed-c2ed9174a4cd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:48.340838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1271/2578] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:07:48 2022] DELETE /identity/v3/projects/25223fe8ff4a4f41b98b5d0aafcd92a9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:48.345871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e411c-1446-4f14-b3a7-f65a34420237 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.346409 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e411c-1446-4f14-b3a7-f65a34420237 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.346527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877e411c-1446-4f14-b3a7-f65a34420237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.369498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-877e411c-1446-4f14-b3a7-f65a34420237 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:48.370887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-877e411c-1446-4f14-b3a7-f65a34420237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:48.444956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1308/2579] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:48.452756 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027359a9-c3ec-42e6-ac2f-a0f17be45414 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.453042 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027359a9-c3ec-42e6-ac2f-a0f17be45414 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.453322 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027359a9-c3ec-42e6-ac2f-a0f17be45414 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.460989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:48.476843 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-027359a9-c3ec-42e6-ac2f-a0f17be45414 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:48.479724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-027359a9-c3ec-42e6-ac2f-a0f17be45414 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:48.487669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1272/2580] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:48.491975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e411c-1446-4f14-b3a7-f65a34420237 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:48.502880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877e411c-1446-4f14-b3a7-f65a34420237 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:48.507750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eafc44-d48f-43e3-887a-f94ef3b697c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:48.508424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eafc44-d48f-43e3-887a-f94ef3b697c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.508836 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eafc44-d48f-43e3-887a-f94ef3b697c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.509624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eafc44-d48f-43e3-887a-f94ef3b697c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.510381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eafc44-d48f-43e3-887a-f94ef3b697c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:07:48.510692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eafc44-d48f-43e3-887a-f94ef3b697c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:48.512302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eafc44-d48f-43e3-887a-f94ef3b697c9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:48.524274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1309/2581] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:07:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:07:48.529457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027359a9-c3ec-42e6-ac2f-a0f17be45414 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:48.539053 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027359a9-c3ec-42e6-ac2f-a0f17be45414 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:48.544521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83f3d2-06a9-40a6-9e20-e2bc6f751b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:48.545152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f3d2-06a9-40a6-9e20-e2bc6f751b67 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.545432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f3d2-06a9-40a6-9e20-e2bc6f751b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.545707 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af83f3d2-06a9-40a6-9e20-e2bc6f751b67 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.546383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f3d2-06a9-40a6-9e20-e2bc6f751b67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:48.548295 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af83f3d2-06a9-40a6-9e20-e2bc6f751b67 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:48.581849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1273/2582] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:07:48 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:48.586116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eafc44-d48f-43e3-887a-f94ef3b697c9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:48.597716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eafc44-d48f-43e3-887a-f94ef3b697c9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:48.656830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecc7dd-a223-4d94-9a2c-55ef2de7a727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:48.657471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecc7dd-a223-4d94-9a2c-55ef2de7a727 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.657749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecc7dd-a223-4d94-9a2c-55ef2de7a727 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.658015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ecc7dd-a223-4d94-9a2c-55ef2de7a727 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.658652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecc7dd-a223-4d94-9a2c-55ef2de7a727 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:48.660593 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ecc7dd-a223-4d94-9a2c-55ef2de7a727 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:48.680253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:48.686803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1310/2583] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:07:48 2022] POST /identity/v3/users => generated 294 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:48.691183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83f3d2-06a9-40a6-9e20-e2bc6f751b67 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:48.701591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af83f3d2-06a9-40a6-9e20-e2bc6f751b67 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:48.709529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80413be8-3ccb-4ec9-8709-8cf8ddb08b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:48.710465 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80413be8-3ccb-4ec9-8709-8cf8ddb08b4d admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.710830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80413be8-3ccb-4ec9-8709-8cf8ddb08b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.711171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80413be8-3ccb-4ec9-8709-8cf8ddb08b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.712126 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80413be8-3ccb-4ec9-8709-8cf8ddb08b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:48.714531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80413be8-3ccb-4ec9-8709-8cf8ddb08b4d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:48.726642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1274/2584] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:48.731421 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecc7dd-a223-4d94-9a2c-55ef2de7a727 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:48.742511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ecc7dd-a223-4d94-9a2c-55ef2de7a727 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:48.748257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088085e4-9dc3-4c95-a217-e98f8f6e1b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:48.748897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088085e4-9dc3-4c95-a217-e98f8f6e1b2e admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.749163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088085e4-9dc3-4c95-a217-e98f8f6e1b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.749425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088085e4-9dc3-4c95-a217-e98f8f6e1b2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.750058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088085e4-9dc3-4c95-a217-e98f8f6e1b2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:48.751688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088085e4-9dc3-4c95-a217-e98f8f6e1b2e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:48.763256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1311/2585] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:48.767656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80413be8-3ccb-4ec9-8709-8cf8ddb08b4d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:48.778853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80413be8-3ccb-4ec9-8709-8cf8ddb08b4d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:48.784000 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8c7c5-85e1-46c7-a9a8-0584c5f6fd57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:48.784784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8c7c5-85e1-46c7-a9a8-0584c5f6fd57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.785061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8c7c5-85e1-46c7-a9a8-0584c5f6fd57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.785332 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c8c7c5-85e1-46c7-a9a8-0584c5f6fd57 admin admin] PATH_INFO: `/v3/projects/43b8906fb2f0485f8398d39e20609d29/users/c5edccdd4d0b4567893cec7c65b9ab38/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.801394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8c7c5-85e1-46c7-a9a8-0584c5f6fd57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=43b8906fb2f0485f8398d39e20609d29, user_id=c5edccdd4d0b4567893cec7c65b9ab38, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:48.804063 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c8c7c5-85e1-46c7-a9a8-0584c5f6fd57 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:48.821385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1275/2586] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:48 2022] PUT /identity/v3/projects/43b8906fb2f0485f8398d39e20609d29/users/c5edccdd4d0b4567893cec7c65b9ab38/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:48.831929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f922f-5b22-4842-82fa-8a10c17dab03 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:48.832216 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f922f-5b22-4842-82fa-8a10c17dab03 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:48.832491 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96f922f-5b22-4842-82fa-8a10c17dab03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:48.862467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f96f922f-5b22-4842-82fa-8a10c17dab03 None None] MFA Rules not processed for user `c5edccdd4d0b4567893cec7c65b9ab38`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:48.863703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f96f922f-5b22-4842-82fa-8a10c17dab03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:48.939453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1312/2587] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:48.945419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8c7c5-85e1-46c7-a9a8-0584c5f6fd57 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:49.013170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c8c7c5-85e1-46c7-a9a8-0584c5f6fd57 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:49.018242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bade3ba-91e4-4808-944e-d4e17f691693 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:49.018868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bade3ba-91e4-4808-944e-d4e17f691693 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.019155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bade3ba-91e4-4808-944e-d4e17f691693 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.019432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bade3ba-91e4-4808-944e-d4e17f691693 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.020109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bade3ba-91e4-4808-944e-d4e17f691693 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:49.022034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bade3ba-91e4-4808-944e-d4e17f691693 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:49.051821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1276/2588] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:07:48 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:07:49.057019 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f922f-5b22-4842-82fa-8a10c17dab03 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:49.068124 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96f922f-5b22-4842-82fa-8a10c17dab03 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:49.132068 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10989259-60b9-4f23-909f-b877258892f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:49.132696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10989259-60b9-4f23-909f-b877258892f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.132971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10989259-60b9-4f23-909f-b877258892f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.133238 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10989259-60b9-4f23-909f-b877258892f4 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.134039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10989259-60b9-4f23-909f-b877258892f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:49.136228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10989259-60b9-4f23-909f-b877258892f4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:49.164482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1313/2589] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:07:49 2022] POST /identity/v3/users => generated 300 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:49.166259 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bade3ba-91e4-4808-944e-d4e17f691693 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:49.176852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bade3ba-91e4-4808-944e-d4e17f691693 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:49.182631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b27b6d-d954-428e-aacd-6b927b870443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:49.183243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b27b6d-d954-428e-aacd-6b927b870443 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.183519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b27b6d-d954-428e-aacd-6b927b870443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.183789 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b27b6d-d954-428e-aacd-6b927b870443 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.184461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b27b6d-d954-428e-aacd-6b927b870443 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:49.186055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b27b6d-d954-428e-aacd-6b927b870443 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:49.196656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1277/2590] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:49.200815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10989259-60b9-4f23-909f-b877258892f4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:49.210811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10989259-60b9-4f23-909f-b877258892f4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:49.215637 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8324f-b746-42f1-af05-0ac14d41aa6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:49.216966 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8324f-b746-42f1-af05-0ac14d41aa6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.217348 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8324f-b746-42f1-af05-0ac14d41aa6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.217702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c8324f-b746-42f1-af05-0ac14d41aa6e admin admin] PATH_INFO: `/v3/projects/1631b0e947b7412082d70488d302e7db/users/fd01566a6cce4a8282a59430ce747819/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.233278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8324f-b746-42f1-af05-0ac14d41aa6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1631b0e947b7412082d70488d302e7db, user_id=fd01566a6cce4a8282a59430ce747819, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:49.236028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c8324f-b746-42f1-af05-0ac14d41aa6e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:49.252427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1314/2591] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:49 2022] PUT /identity/v3/projects/1631b0e947b7412082d70488d302e7db/users/fd01566a6cce4a8282a59430ce747819/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:49.256505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b27b6d-d954-428e-aacd-6b927b870443 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:49.267559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b27b6d-d954-428e-aacd-6b927b870443 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:49.328276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1a54ee8-fef4-4796-985b-ecd40fdd2279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:49.328877 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54ee8-fef4-4796-985b-ecd40fdd2279 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.329150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54ee8-fef4-4796-985b-ecd40fdd2279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.329418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a54ee8-fef4-4796-985b-ecd40fdd2279 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.330048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a54ee8-fef4-4796-985b-ecd40fdd2279 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:49.331700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1a54ee8-fef4-4796-985b-ecd40fdd2279 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:49.341965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1278/2592] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:49.346320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8324f-b746-42f1-af05-0ac14d41aa6e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:49.356947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c8324f-b746-42f1-af05-0ac14d41aa6e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:49.362055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddc8ae-ef8e-491d-aec3-c791831ccb90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:49.362737 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddc8ae-ef8e-491d-aec3-c791831ccb90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.363013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddc8ae-ef8e-491d-aec3-c791831ccb90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.363282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ddc8ae-ef8e-491d-aec3-c791831ccb90 admin admin] PATH_INFO: `/v3/projects/1631b0e947b7412082d70488d302e7db/users/fd01566a6cce4a8282a59430ce747819/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.365291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddc8ae-ef8e-491d-aec3-c791831ccb90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1631b0e947b7412082d70488d302e7db, user_id=fd01566a6cce4a8282a59430ce747819, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:49.367946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ddc8ae-ef8e-491d-aec3-c791831ccb90 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:49.378517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1315/2593] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:49 2022] PUT /identity/v3/projects/1631b0e947b7412082d70488d302e7db/users/fd01566a6cce4a8282a59430ce747819/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:49.390725 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb53516-7958-4512-b7c8-2922e383d8bb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.391033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb53516-7958-4512-b7c8-2922e383d8bb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.391373 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb53516-7958-4512-b7c8-2922e383d8bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.418675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:49.428388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ddb53516-7958-4512-b7c8-2922e383d8bb None None] MFA Rules not processed for user `fd01566a6cce4a8282a59430ce747819`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:49.430819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ddb53516-7958-4512-b7c8-2922e383d8bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:49.502822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1279/2594] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:07:49.514713 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddc8ae-ef8e-491d-aec3-c791831ccb90 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:49.516798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-33ddc8ae-ef8e-491d-aec3-c791831ccb90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:49.553029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:49.579787 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ddc8ae-ef8e-491d-aec3-c791831ccb90 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:49.583725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c991cd68-1b33-4cbe-a3fa-c7de95133271 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:49.584336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991cd68-1b33-4cbe-a3fa-c7de95133271 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.584615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991cd68-1b33-4cbe-a3fa-c7de95133271 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.584887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c991cd68-1b33-4cbe-a3fa-c7de95133271 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.595666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991cd68-1b33-4cbe-a3fa-c7de95133271 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:49.597137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c991cd68-1b33-4cbe-a3fa-c7de95133271 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:49.607559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1316/2595] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:49.637915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb53516-7958-4512-b7c8-2922e383d8bb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:49.709780 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb53516-7958-4512-b7c8-2922e383d8bb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:49.714861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:49.715498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.715745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.716008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] PATH_INFO: `/v3/users/c5edccdd4d0b4567893cec7c65b9ab38` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.717203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5edccdd4d0b4567893cec7c65b9ab38)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:49.719145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:49.772281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5edccdd4d0b4567893cec7c65b9ab38'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:49.781643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5edccdd4d0b4567893cec7c65b9ab38'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:49.793742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5edccdd4d0b4567893cec7c65b9ab38'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:49.794437 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5edccdd4d0b4567893cec7c65b9ab38'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:49.801251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5edccdd4d0b4567893cec7c65b9ab38'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:49.808350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1280/2596] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:07:49 2022] DELETE /identity/v3/users/c5edccdd4d0b4567893cec7c65b9ab38 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:49.813390 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a0436-6780-4b5c-a87f-afd9b6e16521 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.813672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a0436-6780-4b5c-a87f-afd9b6e16521 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.813943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9a0436-6780-4b5c-a87f-afd9b6e16521 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.839450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-cc9a0436-6780-4b5c-a87f-afd9b6e16521 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:49.840666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cc9a0436-6780-4b5c-a87f-afd9b6e16521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:49.899277 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1317/2597] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:49.909984 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:49.912299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:49.930004 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:49.976729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a154bf-c1ae-4086-881a-21b0041f5fcb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:49.980871 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb2e9e-4e9e-4a71-bff1-0f39ef3e7c44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:49.981546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb2e9e-4e9e-4a71-bff1-0f39ef3e7c44 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:49.981782 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb2e9e-4e9e-4a71-bff1-0f39ef3e7c44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:49.982096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedb2e9e-4e9e-4a71-bff1-0f39ef3e7c44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:49.991841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb2e9e-4e9e-4a71-bff1-0f39ef3e7c44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:49.993452 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedb2e9e-4e9e-4a71-bff1-0f39ef3e7c44 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:50.003844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1281/2598] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:50.298477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a0436-6780-4b5c-a87f-afd9b6e16521 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:50.299931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cc9a0436-6780-4b5c-a87f-afd9b6e16521 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:50.313008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9a0436-6780-4b5c-a87f-afd9b6e16521 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:50.319973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:50.320762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:50.321440 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:50.321777 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] PATH_INFO: `/v3/projects/43b8906fb2f0485f8398d39e20609d29` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:50.323710 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=43b8906fb2f0485f8398d39e20609d29)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:50.335463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:50.394912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '43b8906fb2f0485f8398d39e20609d29', 'request_id': 'req-5b888a9f-fb28-4282-b500-4443c09dccf0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.402152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '43b8906fb2f0485f8398d39e20609d29', 'request_id': 'req-5b888a9f-fb28-4282-b500-4443c09dccf0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.408338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '43b8906fb2f0485f8398d39e20609d29', 'request_id': 'req-5b888a9f-fb28-4282-b500-4443c09dccf0'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.425352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] The token cache is being invalidate because project 43b8906fb2f0485f8398d39e20609d29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:07:50.426001 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.427004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.428708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1318/2599] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:07:50 2022] DELETE /identity/v3/projects/43b8906fb2f0485f8398d39e20609d29 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:50.433777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb2e9e-4e9e-4a71-bff1-0f39ef3e7c44 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:50.444038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedb2e9e-4e9e-4a71-bff1-0f39ef3e7c44 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:50.513879 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:50.514700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:50.515016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:50.515293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] PATH_INFO: `/v3/users/fd01566a6cce4a8282a59430ce747819` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:50.516789 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd01566a6cce4a8282a59430ce747819)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:50.519262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:50.584052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd01566a6cce4a8282a59430ce747819'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.595384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd01566a6cce4a8282a59430ce747819'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.617655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd01566a6cce4a8282a59430ce747819'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.618739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd01566a6cce4a8282a59430ce747819'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.625874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-1e44f3e8-0ca2-4ba4-941c-3465d5ac2d97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd01566a6cce4a8282a59430ce747819'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:50.631876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1282/2600] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:07:50 2022] DELETE /identity/v3/users/fd01566a6cce4a8282a59430ce747819 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:50.921992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:50.923557 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:50.937601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b888a9f-fb28-4282-b500-4443c09dccf0 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:50.991120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:50.998438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:50.999092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:50.999382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:50.999673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] PATH_INFO: `/v3/projects/1631b0e947b7412082d70488d302e7db` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.000910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1631b0e947b7412082d70488d302e7db)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.003692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.051741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1631b0e947b7412082d70488d302e7db', 'request_id': 'req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:51.058833 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1631b0e947b7412082d70488d302e7db', 'request_id': 'req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:51.065117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1631b0e947b7412082d70488d302e7db', 'request_id': 'req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:51.081577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] The token cache is being invalidate because project 1631b0e947b7412082d70488d302e7db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:07:51.081869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:51.082602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4df2b9a4-bcc8-4bd7-bc17-95713634f1d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:51.084487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1319/2601] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:07:50 2022] DELETE /identity/v3/projects/1631b0e947b7412082d70488d302e7db => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:51.089416 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe73112-e2bc-430c-86a4-81ebaa74313e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.089709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe73112-e2bc-430c-86a4-81ebaa74313e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.089992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe73112-e2bc-430c-86a4-81ebaa74313e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.110603 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bbe73112-e2bc-430c-86a4-81ebaa74313e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:51.111800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bbe73112-e2bc-430c-86a4-81ebaa74313e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:51.144302 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:51.176293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1283/2602] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:51.184716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deaa85-da43-4149-a1f1-91cd9f338d15 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.184989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deaa85-da43-4149-a1f1-91cd9f338d15 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.185295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22deaa85-da43-4149-a1f1-91cd9f338d15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.205432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-22deaa85-da43-4149-a1f1-91cd9f338d15 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:51.206668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-22deaa85-da43-4149-a1f1-91cd9f338d15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:51.215177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1320/2603] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:51.219794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe73112-e2bc-430c-86a4-81ebaa74313e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:51.230371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe73112-e2bc-430c-86a4-81ebaa74313e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:51.235975 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d507-749b-41f0-b83b-e838a62abe81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:51.236600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d507-749b-41f0-b83b-e838a62abe81 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.236881 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d507-749b-41f0-b83b-e838a62abe81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.237156 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0287d507-749b-41f0-b83b-e838a62abe81 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.237807 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287d507-749b-41f0-b83b-e838a62abe81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:07:51.238125 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287d507-749b-41f0-b83b-e838a62abe81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.239750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0287d507-749b-41f0-b83b-e838a62abe81 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.253002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1284/2604] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:07:51.258030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22deaa85-da43-4149-a1f1-91cd9f338d15 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:51.271405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22deaa85-da43-4149-a1f1-91cd9f338d15 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:51.277341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0ac23-af03-4fd1-a53f-a3dd43618a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:51.277932 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0ac23-af03-4fd1-a53f-a3dd43618a0c admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.278233 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0ac23-af03-4fd1-a53f-a3dd43618a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.278498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa0ac23-af03-4fd1-a53f-a3dd43618a0c admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.279122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0ac23-af03-4fd1-a53f-a3dd43618a0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.281020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa0ac23-af03-4fd1-a53f-a3dd43618a0c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.321367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1321/2605] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:07:51 2022] POST /identity/v3/projects => generated 367 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:07:51.323980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d507-749b-41f0-b83b-e838a62abe81 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:51.334519 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0287d507-749b-41f0-b83b-e838a62abe81 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:51.396128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6011aa-02ca-4d69-8af0-33a63bca4c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:51.396748 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6011aa-02ca-4d69-8af0-33a63bca4c8c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.397030 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6011aa-02ca-4d69-8af0-33a63bca4c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.397301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6011aa-02ca-4d69-8af0-33a63bca4c8c admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.397967 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6011aa-02ca-4d69-8af0-33a63bca4c8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.400203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6011aa-02ca-4d69-8af0-33a63bca4c8c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.425389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1285/2606] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:07:51 2022] POST /identity/v3/users => generated 284 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:51.432003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0ac23-af03-4fd1-a53f-a3dd43618a0c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:51.442932 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa0ac23-af03-4fd1-a53f-a3dd43618a0c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:51.447570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03921f-c4cb-40a2-b263-9fa25211d897 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:51.448201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03921f-c4cb-40a2-b263-9fa25211d897 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.448461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03921f-c4cb-40a2-b263-9fa25211d897 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.448748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03921f-c4cb-40a2-b263-9fa25211d897 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.449380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03921f-c4cb-40a2-b263-9fa25211d897 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.451060 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e03921f-c4cb-40a2-b263-9fa25211d897 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.462426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1322/2607] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:51.466399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6011aa-02ca-4d69-8af0-33a63bca4c8c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:51.477118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6011aa-02ca-4d69-8af0-33a63bca4c8c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:51.482072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5561683-dc4e-4276-b98a-4b8d1075e599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:51.482729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5561683-dc4e-4276-b98a-4b8d1075e599 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.483006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5561683-dc4e-4276-b98a-4b8d1075e599 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.483275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5561683-dc4e-4276-b98a-4b8d1075e599 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.483918 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5561683-dc4e-4276-b98a-4b8d1075e599 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.485517 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5561683-dc4e-4276-b98a-4b8d1075e599 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.499167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1286/2608] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:07:51.500660 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03921f-c4cb-40a2-b263-9fa25211d897 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:51.511628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e03921f-c4cb-40a2-b263-9fa25211d897 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:51.516340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bf39a-5b4c-4844-9436-93f8be62006a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:51.517003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf39a-5b4c-4844-9436-93f8be62006a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.517569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf39a-5b4c-4844-9436-93f8be62006a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.517850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3bf39a-5b4c-4844-9436-93f8be62006a admin admin] PATH_INFO: `/v3/projects/a03402b61698493d9fe26f22b77f2df1/users/a2064ef7e3594e53a4e8f6b3a814a6ce/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.534273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bf39a-5b4c-4844-9436-93f8be62006a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a03402b61698493d9fe26f22b77f2df1, user_id=a2064ef7e3594e53a4e8f6b3a814a6ce, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.536955 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3bf39a-5b4c-4844-9436-93f8be62006a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.551092 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1323/2609] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:51 2022] PUT /identity/v3/projects/a03402b61698493d9fe26f22b77f2df1/users/a2064ef7e3594e53a4e8f6b3a814a6ce/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:51.562949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279e0a0-2f53-4828-aec5-3806ae01a9f2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.563327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279e0a0-2f53-4828-aec5-3806ae01a9f2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.563665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d279e0a0-2f53-4828-aec5-3806ae01a9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.598139 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d279e0a0-2f53-4828-aec5-3806ae01a9f2 None None] MFA Rules not processed for user `a2064ef7e3594e53a4e8f6b3a814a6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:51.599446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d279e0a0-2f53-4828-aec5-3806ae01a9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:51.676356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1287/2610] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:51.682071 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bf39a-5b4c-4844-9436-93f8be62006a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:51.747117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3bf39a-5b4c-4844-9436-93f8be62006a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:51.752101 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b66f0-a2ec-40ad-9b68-e5eda0dc6ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:51.752720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b66f0-a2ec-40ad-9b68-e5eda0dc6ead admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.752995 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b66f0-a2ec-40ad-9b68-e5eda0dc6ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.753265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710b66f0-a2ec-40ad-9b68-e5eda0dc6ead admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.753926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b66f0-a2ec-40ad-9b68-e5eda0dc6ead admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.755873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710b66f0-a2ec-40ad-9b68-e5eda0dc6ead admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.784838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1324/2611] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:07:51 2022] POST /identity/v3/projects => generated 363 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:51.789404 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279e0a0-2f53-4828-aec5-3806ae01a9f2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:51.801870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d279e0a0-2f53-4828-aec5-3806ae01a9f2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:51.847680 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:51.898564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66300e1b-9d2f-454a-aca6-194a24adf0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:51.900399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66300e1b-9d2f-454a-aca6-194a24adf0d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.900399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66300e1b-9d2f-454a-aca6-194a24adf0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.900399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66300e1b-9d2f-454a-aca6-194a24adf0d5 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.901648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66300e1b-9d2f-454a-aca6-194a24adf0d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.904846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66300e1b-9d2f-454a-aca6-194a24adf0d5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.947643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1288/2612] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:07:51 2022] POST /identity/v3/users => generated 288 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:51.953763 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b66f0-a2ec-40ad-9b68-e5eda0dc6ead admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:51.970358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710b66f0-a2ec-40ad-9b68-e5eda0dc6ead admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:51.978063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6e8b2-fee3-45f2-92d8-58588ca8073c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:51.979506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6e8b2-fee3-45f2-92d8-58588ca8073c admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:51.979968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6e8b2-fee3-45f2-92d8-58588ca8073c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:51.980434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a6e8b2-fee3-45f2-92d8-58588ca8073c admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:51.981722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6e8b2-fee3-45f2-92d8-58588ca8073c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:51.983655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a6e8b2-fee3-45f2-92d8-58588ca8073c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:51.987209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:51.999997 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1325/2613] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:52.003861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66300e1b-9d2f-454a-aca6-194a24adf0d5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:52.024067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66300e1b-9d2f-454a-aca6-194a24adf0d5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:52.030385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6e0fb-4346-45bd-a005-82666053170d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:52.031180 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6e0fb-4346-45bd-a005-82666053170d admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:52.031492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6e0fb-4346-45bd-a005-82666053170d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:52.031821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad6e0fb-4346-45bd-a005-82666053170d admin admin] PATH_INFO: `/v3/projects/40339572493a473c96d166be18ced3e1/users/53a549d1c9e14c628d89ddb95b114706/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:52.050874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6e0fb-4346-45bd-a005-82666053170d admin admin] RBAC: Authorizing `identity:create_grant(project_id=40339572493a473c96d166be18ced3e1, user_id=53a549d1c9e14c628d89ddb95b114706, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:52.053617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad6e0fb-4346-45bd-a005-82666053170d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:52.072069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1289/2614] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:52 2022] PUT /identity/v3/projects/40339572493a473c96d166be18ced3e1/users/53a549d1c9e14c628d89ddb95b114706/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:52.075912 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6e8b2-fee3-45f2-92d8-58588ca8073c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:52.087197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a6e8b2-fee3-45f2-92d8-58588ca8073c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:52.143231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb3741-5836-4616-93c2-5ba3de4e9a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:52.143926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb3741-5836-4616-93c2-5ba3de4e9a30 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:52.144094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb3741-5836-4616-93c2-5ba3de4e9a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:52.144393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb3741-5836-4616-93c2-5ba3de4e9a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:52.145036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb3741-5836-4616-93c2-5ba3de4e9a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:52.146707 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb3741-5836-4616-93c2-5ba3de4e9a30 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:52.161446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1326/2615] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:52.162278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6e0fb-4346-45bd-a005-82666053170d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:52.173584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad6e0fb-4346-45bd-a005-82666053170d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:52.179118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd2040-ec45-46df-adb2-d6558e94b4cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:52.179789 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd2040-ec45-46df-adb2-d6558e94b4cb admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:52.180067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd2040-ec45-46df-adb2-d6558e94b4cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:52.180337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cd2040-ec45-46df-adb2-d6558e94b4cb admin admin] PATH_INFO: `/v3/projects/40339572493a473c96d166be18ced3e1/users/53a549d1c9e14c628d89ddb95b114706/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:52.183109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd2040-ec45-46df-adb2-d6558e94b4cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=40339572493a473c96d166be18ced3e1, user_id=53a549d1c9e14c628d89ddb95b114706, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:52.185779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8cd2040-ec45-46df-adb2-d6558e94b4cb admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:52.197216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1290/2616] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:52 2022] PUT /identity/v3/projects/40339572493a473c96d166be18ced3e1/users/53a549d1c9e14c628d89ddb95b114706/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:52.207806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ab124-e61b-4141-a91b-68d3659056af None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:52.208193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ab124-e61b-4141-a91b-68d3659056af None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:52.208552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683ab124-e61b-4141-a91b-68d3659056af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:52.239180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-683ab124-e61b-4141-a91b-68d3659056af None None] MFA Rules not processed for user `53a549d1c9e14c628d89ddb95b114706`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:52.240414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-683ab124-e61b-4141-a91b-68d3659056af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:52.314416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1327/2617] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:52 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:52.327059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd2040-ec45-46df-adb2-d6558e94b4cb admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:52.329091 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b8cd2040-ec45-46df-adb2-d6558e94b4cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:52.396803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cd2040-ec45-46df-adb2-d6558e94b4cb admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:52.401191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e280a17-3eb5-4b51-bcda-2905d88fff24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:52.401824 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e280a17-3eb5-4b51-bcda-2905d88fff24 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:52.402114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e280a17-3eb5-4b51-bcda-2905d88fff24 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:52.402406 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e280a17-3eb5-4b51-bcda-2905d88fff24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:52.412065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e280a17-3eb5-4b51-bcda-2905d88fff24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:52.413511 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e280a17-3eb5-4b51-bcda-2905d88fff24 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:52.423941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1291/2618] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:52 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:52.516550 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ab124-e61b-4141-a91b-68d3659056af None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:52.526939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683ab124-e61b-4141-a91b-68d3659056af None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:52.531009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2258656b-0466-404f-b104-91e9e4e85a00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:52.531708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258656b-0466-404f-b104-91e9e4e85a00 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:52.532054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258656b-0466-404f-b104-91e9e4e85a00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:52.532396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2258656b-0466-404f-b104-91e9e4e85a00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:52.596811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258656b-0466-404f-b104-91e9e4e85a00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:52.596811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2258656b-0466-404f-b104-91e9e4e85a00 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:52.608635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1328/2619] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:57.473971 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e280a17-3eb5-4b51-bcda-2905d88fff24 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:57.548635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e280a17-3eb5-4b51-bcda-2905d88fff24 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:57.553701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:57.554620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:57.556045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:57.556045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] PATH_INFO: `/v3/users/a2064ef7e3594e53a4e8f6b3a814a6ce` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:57.557525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2064ef7e3594e53a4e8f6b3a814a6ce)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:57.560740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:57.604267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:57.636849 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2064ef7e3594e53a4e8f6b3a814a6ce'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:57.648438 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2064ef7e3594e53a4e8f6b3a814a6ce'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:57.665723 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2064ef7e3594e53a4e8f6b3a814a6ce'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:57.666912 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2064ef7e3594e53a4e8f6b3a814a6ce'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:57.679588 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2064ef7e3594e53a4e8f6b3a814a6ce'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:57.691769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1292/2620] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:07:57 2022] DELETE /identity/v3/users/a2064ef7e3594e53a4e8f6b3a814a6ce => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:57.699191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27c7a0-47ed-47c0-978d-fe5d9bbb89e5 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:57.699758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27c7a0-47ed-47c0-978d-fe5d9bbb89e5 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:57.699758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c27c7a0-47ed-47c0-978d-fe5d9bbb89e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:57.720055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6c27c7a0-47ed-47c0-978d-fe5d9bbb89e5 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:57.721278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6c27c7a0-47ed-47c0-978d-fe5d9bbb89e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:57.735078 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:57.786497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1329/2621] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:57.796055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:57.797487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:57.858227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c2660c-832d-4ba2-8f2e-29db53155ab9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:57.862658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0481f-bb1c-4279-ae93-64f3d1bf0ebc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:57.863290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0481f-bb1c-4279-ae93-64f3d1bf0ebc service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:57.863597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0481f-bb1c-4279-ae93-64f3d1bf0ebc service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:57.863872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab0481f-bb1c-4279-ae93-64f3d1bf0ebc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:57.874608 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0481f-bb1c-4279-ae93-64f3d1bf0ebc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:57.876265 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab0481f-bb1c-4279-ae93-64f3d1bf0ebc service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:57.886882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1293/2622] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:07:58.177023 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27c7a0-47ed-47c0-978d-fe5d9bbb89e5 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:58.178616 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6c27c7a0-47ed-47c0-978d-fe5d9bbb89e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:58.192957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c27c7a0-47ed-47c0-978d-fe5d9bbb89e5 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:58.198649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:58.199423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:58.199684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:58.199953 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] PATH_INFO: `/v3/projects/a03402b61698493d9fe26f22b77f2df1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:58.201151 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a03402b61698493d9fe26f22b77f2df1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:58.203102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:58.253726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a03402b61698493d9fe26f22b77f2df1', 'request_id': 'req-5df07614-12fc-4b33-9032-cff94c640a55'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.261720 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a03402b61698493d9fe26f22b77f2df1', 'request_id': 'req-5df07614-12fc-4b33-9032-cff94c640a55'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.267908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a03402b61698493d9fe26f22b77f2df1', 'request_id': 'req-5df07614-12fc-4b33-9032-cff94c640a55'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.283433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] The token cache is being invalidate because project a03402b61698493d9fe26f22b77f2df1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:07:58.283728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.284356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.285930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1330/2623] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:07:58 2022] DELETE /identity/v3/projects/a03402b61698493d9fe26f22b77f2df1 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:58.290245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0481f-bb1c-4279-ae93-64f3d1bf0ebc service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:58.300689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab0481f-bb1c-4279-ae93-64f3d1bf0ebc service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:58.374965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:58.375579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:58.376070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:58.376384 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] PATH_INFO: `/v3/users/53a549d1c9e14c628d89ddb95b114706` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:58.378114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=53a549d1c9e14c628d89ddb95b114706)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:58.381687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:58.459229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53a549d1c9e14c628d89ddb95b114706'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.462288 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:58.470636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53a549d1c9e14c628d89ddb95b114706'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.485252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53a549d1c9e14c628d89ddb95b114706'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.486163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53a549d1c9e14c628d89ddb95b114706'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.493235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6df8e43c-4272-4ef3-8e6a-cca34bf84f6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53a549d1c9e14c628d89ddb95b114706'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.499186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1294/2624] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:07:58 2022] DELETE /identity/v3/users/53a549d1c9e14c628d89ddb95b114706 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:58.800293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:58.801859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:58.817028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df07614-12fc-4b33-9032-cff94c640a55 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:58.875827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:58.876469 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:58.876744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:58.877017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] PATH_INFO: `/v3/projects/40339572493a473c96d166be18ced3e1` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:58.878144 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] RBAC: Authorizing `identity:delete_project(project_id=40339572493a473c96d166be18ced3e1)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:58.880093 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:58.927022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:58.931197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40339572493a473c96d166be18ced3e1', 'request_id': 'req-97d0bff2-bfcf-450b-aae5-9b63045ac30b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.940273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40339572493a473c96d166be18ced3e1', 'request_id': 'req-97d0bff2-bfcf-450b-aae5-9b63045ac30b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.948713 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40339572493a473c96d166be18ced3e1', 'request_id': 'req-97d0bff2-bfcf-450b-aae5-9b63045ac30b'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.961976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] The token cache is being invalidate because project 40339572493a473c96d166be18ced3e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:07:58.962251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.962826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-97d0bff2-bfcf-450b-aae5-9b63045ac30b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:07:58.964329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1331/2625] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:07:58 2022] DELETE /identity/v3/projects/40339572493a473c96d166be18ced3e1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:58.969457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef653e1-768a-429a-868f-5dd2f50af7a9 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:58.969745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef653e1-768a-429a-868f-5dd2f50af7a9 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:58.970020 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef653e1-768a-429a-868f-5dd2f50af7a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:58.990271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-eef653e1-768a-429a-868f-5dd2f50af7a9 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:58.991516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-eef653e1-768a-429a-868f-5dd2f50af7a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:59.077444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1295/2626] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:59.084760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e8403-f631-4065-a1fe-0a9628ac70b7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:59.085051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e8403-f631-4065-a1fe-0a9628ac70b7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:59.085328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e8403-f631-4065-a1fe-0a9628ac70b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:59.107069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-c79e8403-f631-4065-a1fe-0a9628ac70b7 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:59.108340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c79e8403-f631-4065-a1fe-0a9628ac70b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:59.116308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1332/2627] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:59.120973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef653e1-768a-429a-868f-5dd2f50af7a9 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:59.131129 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef653e1-768a-429a-868f-5dd2f50af7a9 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:59.136798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44d3a3-be1d-48b3-87ad-e0f00a3fd94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:59.137440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44d3a3-be1d-48b3-87ad-e0f00a3fd94c admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:59.137718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44d3a3-be1d-48b3-87ad-e0f00a3fd94c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:59.138110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e44d3a3-be1d-48b3-87ad-e0f00a3fd94c admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:59.138766 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44d3a3-be1d-48b3-87ad-e0f00a3fd94c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:07:59.139073 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44d3a3-be1d-48b3-87ad-e0f00a3fd94c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:59.141089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e44d3a3-be1d-48b3-87ad-e0f00a3fd94c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:59.154255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1296/2628] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:07:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:07:59.158759 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e8403-f631-4065-a1fe-0a9628ac70b7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:59.169942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e8403-f631-4065-a1fe-0a9628ac70b7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:59.175744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0517fc2-f417-49a1-96f2-b59eeef2a3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:59.176370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0517fc2-f417-49a1-96f2-b59eeef2a3fa admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:59.176648 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0517fc2-f417-49a1-96f2-b59eeef2a3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:59.176922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0517fc2-f417-49a1-96f2-b59eeef2a3fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:59.177553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0517fc2-f417-49a1-96f2-b59eeef2a3fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:59.179503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0517fc2-f417-49a1-96f2-b59eeef2a3fa admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:59.212692 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1333/2629] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:07:59 2022] POST /identity/v3/projects => generated 375 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:07:59.216875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44d3a3-be1d-48b3-87ad-e0f00a3fd94c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:59.227634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e44d3a3-be1d-48b3-87ad-e0f00a3fd94c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:59.297551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acc4a6-1477-4325-95b8-a783be020fc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:59.298226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acc4a6-1477-4325-95b8-a783be020fc9 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:59.298499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acc4a6-1477-4325-95b8-a783be020fc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:59.298806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01acc4a6-1477-4325-95b8-a783be020fc9 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:59.299442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acc4a6-1477-4325-95b8-a783be020fc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:59.301402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01acc4a6-1477-4325-95b8-a783be020fc9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:59.327157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1297/2630] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:07:59 2022] POST /identity/v3/users => generated 288 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:07:59.332000 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0517fc2-f417-49a1-96f2-b59eeef2a3fa admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:59.343420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0517fc2-f417-49a1-96f2-b59eeef2a3fa admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:59.350549 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033fe6d-07dd-4168-a1a3-7e6b55d66e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:59.351418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033fe6d-07dd-4168-a1a3-7e6b55d66e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:59.351826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033fe6d-07dd-4168-a1a3-7e6b55d66e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:59.352252 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5033fe6d-07dd-4168-a1a3-7e6b55d66e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:59.353010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033fe6d-07dd-4168-a1a3-7e6b55d66e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:59.355207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5033fe6d-07dd-4168-a1a3-7e6b55d66e5b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:59.365853 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1334/2631] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:07:59.370663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acc4a6-1477-4325-95b8-a783be020fc9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:59.381755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01acc4a6-1477-4325-95b8-a783be020fc9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:59.387639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a7c58e-0035-4b7c-8a52-7c95683efb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:59.388293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7c58e-0035-4b7c-8a52-7c95683efb4f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:59.388572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7c58e-0035-4b7c-8a52-7c95683efb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:59.388845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a7c58e-0035-4b7c-8a52-7c95683efb4f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:59.389499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7c58e-0035-4b7c-8a52-7c95683efb4f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:59.391167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a7c58e-0035-4b7c-8a52-7c95683efb4f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:59.402701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1298/2632] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:07:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:07:59.407145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033fe6d-07dd-4168-a1a3-7e6b55d66e5b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:59.418109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5033fe6d-07dd-4168-a1a3-7e6b55d66e5b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:59.423257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590595c0-9d6e-4d1b-b961-7380350197fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:59.423892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590595c0-9d6e-4d1b-b961-7380350197fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:59.424200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590595c0-9d6e-4d1b-b961-7380350197fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:59.424473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590595c0-9d6e-4d1b-b961-7380350197fb admin admin] PATH_INFO: `/v3/projects/4cbf8e7814964ffe895b44787b650283/users/72f1afc32fee449789afa49d68e1f954/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:59.440847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590595c0-9d6e-4d1b-b961-7380350197fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cbf8e7814964ffe895b44787b650283, user_id=72f1afc32fee449789afa49d68e1f954, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:59.443525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590595c0-9d6e-4d1b-b961-7380350197fb admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:59.458492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1335/2633] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:07:59 2022] PUT /identity/v3/projects/4cbf8e7814964ffe895b44787b650283/users/72f1afc32fee449789afa49d68e1f954/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:07:59.469937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fadc7-f948-4a3d-b721-d88b909981be None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:59.470285 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fadc7-f948-4a3d-b721-d88b909981be None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:59.470651 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34fadc7-f948-4a3d-b721-d88b909981be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:59.499597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b34fadc7-f948-4a3d-b721-d88b909981be None None] MFA Rules not processed for user `72f1afc32fee449789afa49d68e1f954`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:07:59.500840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b34fadc7-f948-4a3d-b721-d88b909981be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:59.523132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:07:59.572210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1299/2634] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:07:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:07:59.583703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590595c0-9d6e-4d1b-b961-7380350197fb admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:07:59.585489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-590595c0-9d6e-4d1b-b961-7380350197fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:07:59.648484 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590595c0-9d6e-4d1b-b961-7380350197fb admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:07:59.652182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49142fd1-448f-4c3e-8448-9b3f1b96966b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3505c8965c4a42e493a2f5916cab56dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:07:59.653075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49142fd1-448f-4c3e-8448-9b3f1b96966b service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:07:59.653199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49142fd1-448f-4c3e-8448-9b3f1b96966b service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:07:59.653361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49142fd1-448f-4c3e-8448-9b3f1b96966b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:07:59.662160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49142fd1-448f-4c3e-8448-9b3f1b96966b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:07:59.663610 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49142fd1-448f-4c3e-8448-9b3f1b96966b service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:07:59.673607 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1336/2635] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:07:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:08:09.569488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=17 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:09.669152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=61,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:17.470069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fadc7-f948-4a3d-b721-d88b909981be None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:17.539354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34fadc7-f948-4a3d-b721-d88b909981be None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:17.544441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:17.545080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:17.545362 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:17.545639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] PATH_INFO: `/v3/users/72f1afc32fee449789afa49d68e1f954` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:17.546834 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72f1afc32fee449789afa49d68e1f954)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:17.548767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:17.603620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72f1afc32fee449789afa49d68e1f954'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:17.612477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72f1afc32fee449789afa49d68e1f954'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:17.625026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72f1afc32fee449789afa49d68e1f954'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:17.625665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72f1afc32fee449789afa49d68e1f954'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:17.632644 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72f1afc32fee449789afa49d68e1f954'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:17.638667 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1300/2636] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:08:17 2022] DELETE /identity/v3/users/72f1afc32fee449789afa49d68e1f954 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:17.643302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d210be-2f52-41d6-9c2c-054341357ec8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:17.643828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d210be-2f52-41d6-9c2c-054341357ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:17.643828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d210be-2f52-41d6-9c2c-054341357ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:17.664634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b4d210be-2f52-41d6-9c2c-054341357ec8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:17.665878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b4d210be-2f52-41d6-9c2c-054341357ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:17.725513 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1337/2637] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:17.736328 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:17.738836 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:17.809828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ce9923-d66d-495f-91ea-4c437b808887 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:17.814607 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea3b560-cf27-4c61-9fe9-25907458116a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:17.815234 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3b560-cf27-4c61-9fe9-25907458116a service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:17.815514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3b560-cf27-4c61-9fe9-25907458116a service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:17.815796 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea3b560-cf27-4c61-9fe9-25907458116a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:17.826941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3b560-cf27-4c61-9fe9-25907458116a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:17.828392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea3b560-cf27-4c61-9fe9-25907458116a service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:17.838856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1301/2638] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:08:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:08:18.159976 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d210be-2f52-41d6-9c2c-054341357ec8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:18.161512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b4d210be-2f52-41d6-9c2c-054341357ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:18.175006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d210be-2f52-41d6-9c2c-054341357ec8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:18.181055 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:18.181739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.182047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.182441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] PATH_INFO: `/v3/projects/4cbf8e7814964ffe895b44787b650283` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.183615 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cbf8e7814964ffe895b44787b650283)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:18.185556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:18.232227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cbf8e7814964ffe895b44787b650283', 'request_id': 'req-4e86bc4d-3aee-4917-95b6-6b76ca235e85'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:18.238178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cbf8e7814964ffe895b44787b650283', 'request_id': 'req-4e86bc4d-3aee-4917-95b6-6b76ca235e85'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:18.243840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cbf8e7814964ffe895b44787b650283', 'request_id': 'req-4e86bc4d-3aee-4917-95b6-6b76ca235e85'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:18.258687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] The token cache is being invalidate because project 4cbf8e7814964ffe895b44787b650283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:08:18.258956 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:18.259606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4e86bc4d-3aee-4917-95b6-6b76ca235e85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:18.261140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1338/2639] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:08:18 2022] DELETE /identity/v3/projects/4cbf8e7814964ffe895b44787b650283 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:18.267035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930c9c6-f79a-4ea6-b050-2b263606f4d3 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.267305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930c9c6-f79a-4ea6-b050-2b263606f4d3 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.267589 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7930c9c6-f79a-4ea6-b050-2b263606f4d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.288637 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-7930c9c6-f79a-4ea6-b050-2b263606f4d3 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:18.289846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7930c9c6-f79a-4ea6-b050-2b263606f4d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:18.308538 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:18.355708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1302/2640] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:08:18.367512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73208427-81de-4b0a-944c-4c7a32074299 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.367791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73208427-81de-4b0a-944c-4c7a32074299 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.368058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73208427-81de-4b0a-944c-4c7a32074299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.392199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-73208427-81de-4b0a-944c-4c7a32074299 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:18.393811 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-73208427-81de-4b0a-944c-4c7a32074299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:18.405439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1339/2641] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:18.409926 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930c9c6-f79a-4ea6-b050-2b263606f4d3 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:18.420252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7930c9c6-f79a-4ea6-b050-2b263606f4d3 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:18.425956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc50ed3-5a24-485d-88a6-60bc63fb9bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:18.426600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc50ed3-5a24-485d-88a6-60bc63fb9bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.426883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc50ed3-5a24-485d-88a6-60bc63fb9bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.427150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc50ed3-5a24-485d-88a6-60bc63fb9bd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.427798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc50ed3-5a24-485d-88a6-60bc63fb9bd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:08:18.428099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc50ed3-5a24-485d-88a6-60bc63fb9bd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:18.429712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc50ed3-5a24-485d-88a6-60bc63fb9bd4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:18.442665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1303/2642] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:08:18.447122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73208427-81de-4b0a-944c-4c7a32074299 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:18.457231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73208427-81de-4b0a-944c-4c7a32074299 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:18.462386 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c37afc-9269-44e7-85be-73ae87039911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:18.463047 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c37afc-9269-44e7-85be-73ae87039911 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.463328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c37afc-9269-44e7-85be-73ae87039911 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.463602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c37afc-9269-44e7-85be-73ae87039911 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.464259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c37afc-9269-44e7-85be-73ae87039911 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:18.466209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c37afc-9269-44e7-85be-73ae87039911 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:18.500105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1340/2643] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:08:18 2022] POST /identity/v3/projects => generated 369 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:18.501555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc50ed3-5a24-485d-88a6-60bc63fb9bd4 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:18.512149 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc50ed3-5a24-485d-88a6-60bc63fb9bd4 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:18.575399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed59ce2-a588-433e-bd5c-18bbd610a6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:18.576160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed59ce2-a588-433e-bd5c-18bbd610a6be admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.576446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed59ce2-a588-433e-bd5c-18bbd610a6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.576722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed59ce2-a588-433e-bd5c-18bbd610a6be admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.577364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed59ce2-a588-433e-bd5c-18bbd610a6be admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:18.579915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed59ce2-a588-433e-bd5c-18bbd610a6be admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:18.613665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1304/2644] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:08:18 2022] POST /identity/v3/users => generated 285 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:18.618291 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c37afc-9269-44e7-85be-73ae87039911 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:18.630473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c37afc-9269-44e7-85be-73ae87039911 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:18.636467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf72f6-9354-44b3-a289-2f7186fb7504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:18.637217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf72f6-9354-44b3-a289-2f7186fb7504 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.637566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf72f6-9354-44b3-a289-2f7186fb7504 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.637910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf72f6-9354-44b3-a289-2f7186fb7504 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.639006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf72f6-9354-44b3-a289-2f7186fb7504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:18.640803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf72f6-9354-44b3-a289-2f7186fb7504 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:18.657140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1341/2645] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:18.658515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed59ce2-a588-433e-bd5c-18bbd610a6be admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:18.670343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed59ce2-a588-433e-bd5c-18bbd610a6be admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:18.675615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a546fa-4bf8-4ae6-bc6d-4f7c9034c942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:18.676260 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a546fa-4bf8-4ae6-bc6d-4f7c9034c942 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.676543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a546fa-4bf8-4ae6-bc6d-4f7c9034c942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.676816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a546fa-4bf8-4ae6-bc6d-4f7c9034c942 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.678419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a546fa-4bf8-4ae6-bc6d-4f7c9034c942 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:18.680835 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a546fa-4bf8-4ae6-bc6d-4f7c9034c942 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:18.692882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1305/2646] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:18.697569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf72f6-9354-44b3-a289-2f7186fb7504 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:18.708751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf72f6-9354-44b3-a289-2f7186fb7504 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:18.714715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6155-8e3b-49ed-9415-ee05a280acff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:18.715436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6155-8e3b-49ed-9415-ee05a280acff admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.715715 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6155-8e3b-49ed-9415-ee05a280acff admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.716040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b6155-8e3b-49ed-9415-ee05a280acff admin admin] PATH_INFO: `/v3/projects/f4f30c8da851457f96e1f2a309a14769/users/8c7aca22245745c49b13d1265c920e96/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.733259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b6155-8e3b-49ed-9415-ee05a280acff admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f30c8da851457f96e1f2a309a14769, user_id=8c7aca22245745c49b13d1265c920e96, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:18.735987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d51b6155-8e3b-49ed-9415-ee05a280acff admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:18.752382 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1342/2647] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:08:18 2022] PUT /identity/v3/projects/f4f30c8da851457f96e1f2a309a14769/users/8c7aca22245745c49b13d1265c920e96/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:18.765056 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea9958-2bef-414e-92a8-39058c5d3947 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.765365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea9958-2bef-414e-92a8-39058c5d3947 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.765620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ea9958-2bef-414e-92a8-39058c5d3947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.799287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-57ea9958-2bef-414e-92a8-39058c5d3947 None None] MFA Rules not processed for user `8c7aca22245745c49b13d1265c920e96`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:18.801262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-57ea9958-2bef-414e-92a8-39058c5d3947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:18.876146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1306/2648] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:18 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:18.888860 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6155-8e3b-49ed-9415-ee05a280acff admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:18.890739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d51b6155-8e3b-49ed-9415-ee05a280acff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:18.913605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:18.955276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b6155-8e3b-49ed-9415-ee05a280acff admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:18.959668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75db5c-dead-41a3-81d6-569399e3d42b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3505c8965c4a42e493a2f5916cab56dc', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:18.960319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75db5c-dead-41a3-81d6-569399e3d42b service glance] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:18.960588 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75db5c-dead-41a3-81d6-569399e3d42b service glance] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:18.960922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b75db5c-dead-41a3-81d6-569399e3d42b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:18.970017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75db5c-dead-41a3-81d6-569399e3d42b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:18.971601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b75db5c-dead-41a3-81d6-569399e3d42b service glance] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:18.980931 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1343/2649] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:08:18 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:08:26.950283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea9958-2bef-414e-92a8-39058c5d3947 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:26.951775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-57ea9958-2bef-414e-92a8-39058c5d3947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:26.973869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:27.014586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ea9958-2bef-414e-92a8-39058c5d3947 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:27.018703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb852a4b-9454-4540-9a9a-dfa282ea3148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:27.019311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb852a4b-9454-4540-9a9a-dfa282ea3148 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:27.019585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb852a4b-9454-4540-9a9a-dfa282ea3148 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:27.019853 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb852a4b-9454-4540-9a9a-dfa282ea3148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:27.021854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bb852a4b-9454-4540-9a9a-dfa282ea3148 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:27.092479 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb852a4b-9454-4540-9a9a-dfa282ea3148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:27.093943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb852a4b-9454-4540-9a9a-dfa282ea3148 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:27.104309 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1307/2650] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:08:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:08:28.976556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=17 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:30.548479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75db5c-dead-41a3-81d6-569399e3d42b service glance] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:30.550436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8b75db5c-dead-41a3-81d6-569399e3d42b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:30.567475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b75db5c-dead-41a3-81d6-569399e3d42b service glance] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:30.572129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:30.572835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:30.573115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:30.573864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] PATH_INFO: `/v3/users/ecf2dfd9bbe54b3085e9638caf8ad044` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:30.575066 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecf2dfd9bbe54b3085e9638caf8ad044)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:30.576996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:30.634242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecf2dfd9bbe54b3085e9638caf8ad044'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:30.634885 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecf2dfd9bbe54b3085e9638caf8ad044'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:30.643678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecf2dfd9bbe54b3085e9638caf8ad044'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:30.655320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecf2dfd9bbe54b3085e9638caf8ad044'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:30.655952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-71df6735-b6e0-464a-8ffb-4613c6706ca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecf2dfd9bbe54b3085e9638caf8ad044'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:30.667922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1344/2651] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:08:30 2022] DELETE /identity/v3/users/ecf2dfd9bbe54b3085e9638caf8ad044 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:30.844460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb852a4b-9454-4540-9a9a-dfa282ea3148 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:30.855022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb852a4b-9454-4540-9a9a-dfa282ea3148 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:30.918269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:30.918904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:30.919178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:30.919448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] PATH_INFO: `/v3/projects/5d5ac5d4506943c48f4f7640d97f63c5` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:30.920575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d5ac5d4506943c48f4f7640d97f63c5)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:30.922515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:30.976165 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d5ac5d4506943c48f4f7640d97f63c5', 'request_id': 'req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:30.982675 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d5ac5d4506943c48f4f7640d97f63c5', 'request_id': 'req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:30.989232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d5ac5d4506943c48f4f7640d97f63c5', 'request_id': 'req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:30.999008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:31.007401 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] The token cache is being invalidate because project 5d5ac5d4506943c48f4f7640d97f63c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:08:31.007791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:31.009041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0378eb7e-78cd-4a2c-bc0a-bbb1579f4f8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:31.011571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1308/2652] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:08:30 2022] DELETE /identity/v3/projects/5d5ac5d4506943c48f4f7640d97f63c5 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:31.016687 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9967e97-d746-4f78-b492-1f6e0e8a68b3 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.016973 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9967e97-d746-4f78-b492-1f6e0e8a68b3 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.017267 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9967e97-d746-4f78-b492-1f6e0e8a68b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.039046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a9967e97-d746-4f78-b492-1f6e0e8a68b3 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:31.040275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a9967e97-d746-4f78-b492-1f6e0e8a68b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:31.102183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1345/2653] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:31.110244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bea066-d888-4c1e-a400-8b0d2059658e None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.110534 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bea066-d888-4c1e-a400-8b0d2059658e None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.110813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bea066-d888-4c1e-a400-8b0d2059658e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.131555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-b1bea066-d888-4c1e-a400-8b0d2059658e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:31.132774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b1bea066-d888-4c1e-a400-8b0d2059658e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:31.141377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1309/2654] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:31.146512 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9967e97-d746-4f78-b492-1f6e0e8a68b3 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:31.160926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9967e97-d746-4f78-b492-1f6e0e8a68b3 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:31.166099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b43b1b-64dd-4390-a05b-a988fafbd278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:31.166718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b43b1b-64dd-4390-a05b-a988fafbd278 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.166996 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b43b1b-64dd-4390-a05b-a988fafbd278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.167272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b43b1b-64dd-4390-a05b-a988fafbd278 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.167915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b43b1b-64dd-4390-a05b-a988fafbd278 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:08:31.168223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b43b1b-64dd-4390-a05b-a988fafbd278 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:31.169834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b43b1b-64dd-4390-a05b-a988fafbd278 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:31.184401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1346/2655] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:08:31.191242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bea066-d888-4c1e-a400-8b0d2059658e None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:31.206108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bea066-d888-4c1e-a400-8b0d2059658e None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:31.212514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ba321-8f40-47d8-8d14-b0cfc31808a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:31.213122 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ba321-8f40-47d8-8d14-b0cfc31808a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.213393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ba321-8f40-47d8-8d14-b0cfc31808a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.213659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9ba321-8f40-47d8-8d14-b0cfc31808a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.214319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ba321-8f40-47d8-8d14-b0cfc31808a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:31.216992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9ba321-8f40-47d8-8d14-b0cfc31808a1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:31.247094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1310/2656] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:08:31 2022] POST /identity/v3/projects => generated 403 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:31.251381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b43b1b-64dd-4390-a05b-a988fafbd278 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:31.261450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b43b1b-64dd-4390-a05b-a988fafbd278 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:31.322211 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ac4a2-5658-43b7-836e-a7d1f0f1d7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:31.322825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ac4a2-5658-43b7-836e-a7d1f0f1d7a1 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.323104 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ac4a2-5658-43b7-836e-a7d1f0f1d7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.323375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ac4a2-5658-43b7-836e-a7d1f0f1d7a1 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.324011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ac4a2-5658-43b7-836e-a7d1f0f1d7a1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:31.325934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ac4a2-5658-43b7-836e-a7d1f0f1d7a1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:31.334100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:31.352337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1347/2657] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:08:31 2022] POST /identity/v3/users => generated 302 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:31.356421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ba321-8f40-47d8-8d14-b0cfc31808a1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:31.366708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9ba321-8f40-47d8-8d14-b0cfc31808a1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:31.371830 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7401ec9-e415-478d-be75-db0f0142800b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:31.372435 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7401ec9-e415-478d-be75-db0f0142800b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.372709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7401ec9-e415-478d-be75-db0f0142800b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.372981 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7401ec9-e415-478d-be75-db0f0142800b admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.373659 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7401ec9-e415-478d-be75-db0f0142800b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:31.375293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7401ec9-e415-478d-be75-db0f0142800b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:31.386021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1311/2658] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:31.390297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ac4a2-5658-43b7-836e-a7d1f0f1d7a1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:31.401396 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ac4a2-5658-43b7-836e-a7d1f0f1d7a1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:31.407025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bddde0-38dc-4da4-97da-36342b49b9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:31.407626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bddde0-38dc-4da4-97da-36342b49b9b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.407902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bddde0-38dc-4da4-97da-36342b49b9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.408225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bddde0-38dc-4da4-97da-36342b49b9b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.408871 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bddde0-38dc-4da4-97da-36342b49b9b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:31.410528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bddde0-38dc-4da4-97da-36342b49b9b7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:31.421223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1348/2659] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:31.426512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7401ec9-e415-478d-be75-db0f0142800b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:31.440229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7401ec9-e415-478d-be75-db0f0142800b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:31.448185 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76573c1-758d-4caa-b761-57aed0cb5ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:31.449211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76573c1-758d-4caa-b761-57aed0cb5ed3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.449585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76573c1-758d-4caa-b761-57aed0cb5ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.449964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76573c1-758d-4caa-b761-57aed0cb5ed3 admin admin] PATH_INFO: `/v3/projects/e7bedd1544c64b8d9f06c23c64ec7c50/users/c4250e2f4bce47daa703bc1b67f8b11f/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.467813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76573c1-758d-4caa-b761-57aed0cb5ed3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bedd1544c64b8d9f06c23c64ec7c50, user_id=c4250e2f4bce47daa703bc1b67f8b11f, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:31.470567 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76573c1-758d-4caa-b761-57aed0cb5ed3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:31.486061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1312/2660] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:08:31 2022] PUT /identity/v3/projects/e7bedd1544c64b8d9f06c23c64ec7c50/users/c4250e2f4bce47daa703bc1b67f8b11f/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:31.496805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7123883-51cf-429a-bee0-2a0dc601b8d8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.497090 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7123883-51cf-429a-bee0-2a0dc601b8d8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.497418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7123883-51cf-429a-bee0-2a0dc601b8d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.529946 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a7123883-51cf-429a-bee0-2a0dc601b8d8 None None] MFA Rules not processed for user `c4250e2f4bce47daa703bc1b67f8b11f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:31.531219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a7123883-51cf-429a-bee0-2a0dc601b8d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:31.741894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1349/2661] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:31 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:08:31.746728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76573c1-758d-4caa-b761-57aed0cb5ed3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:31.815456 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76573c1-758d-4caa-b761-57aed0cb5ed3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:31.821021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48630f-89d5-47af-aa9a-8b81538887f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:31.821643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48630f-89d5-47af-aa9a-8b81538887f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.821921 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48630f-89d5-47af-aa9a-8b81538887f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.822227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f48630f-89d5-47af-aa9a-8b81538887f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.822859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48630f-89d5-47af-aa9a-8b81538887f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:31.824767 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f48630f-89d5-47af-aa9a-8b81538887f6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:31.855149 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1313/2662] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:08:31 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:31.859407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7123883-51cf-429a-bee0-2a0dc601b8d8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:31.873774 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7123883-51cf-429a-bee0-2a0dc601b8d8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:31.935487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18a769-6498-45a2-96d8-7cd97f617c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:31.936099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18a769-6498-45a2-96d8-7cd97f617c61 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.936380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18a769-6498-45a2-96d8-7cd97f617c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.936654 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18a769-6498-45a2-96d8-7cd97f617c61 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.937270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18a769-6498-45a2-96d8-7cd97f617c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:31.939228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18a769-6498-45a2-96d8-7cd97f617c61 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:31.962552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1350/2663] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:08:31 2022] POST /identity/v3/users => generated 309 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:31.966782 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48630f-89d5-47af-aa9a-8b81538887f6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:31.977702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f48630f-89d5-47af-aa9a-8b81538887f6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:31.983310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dc2b5-4a5c-4cae-9dfd-8c8d1ff91615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:31.983917 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dc2b5-4a5c-4cae-9dfd-8c8d1ff91615 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:31.984195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dc2b5-4a5c-4cae-9dfd-8c8d1ff91615 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:31.984462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0dc2b5-4a5c-4cae-9dfd-8c8d1ff91615 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:31.985104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0dc2b5-4a5c-4cae-9dfd-8c8d1ff91615 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:31.987006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0dc2b5-4a5c-4cae-9dfd-8c8d1ff91615 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:31.998142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1314/2664] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:32.002568 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18a769-6498-45a2-96d8-7cd97f617c61 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:32.013302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18a769-6498-45a2-96d8-7cd97f617c61 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:32.019373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf707ed-c35c-41fe-97bd-d9de7fdff978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:32.020035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf707ed-c35c-41fe-97bd-d9de7fdff978 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:32.020313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf707ed-c35c-41fe-97bd-d9de7fdff978 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:32.020583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf707ed-c35c-41fe-97bd-d9de7fdff978 admin admin] PATH_INFO: `/v3/projects/d3d9417f5c184f72acf09c2af39a7dff/users/e17b54e8f23b49508a0ef15118610566/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:32.037653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf707ed-c35c-41fe-97bd-d9de7fdff978 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d9417f5c184f72acf09c2af39a7dff, user_id=e17b54e8f23b49508a0ef15118610566, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:32.040387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf707ed-c35c-41fe-97bd-d9de7fdff978 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:32.056975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1351/2665] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:08:32 2022] PUT /identity/v3/projects/d3d9417f5c184f72acf09c2af39a7dff/users/e17b54e8f23b49508a0ef15118610566/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:32.061848 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dc2b5-4a5c-4cae-9dfd-8c8d1ff91615 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:32.076743 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0dc2b5-4a5c-4cae-9dfd-8c8d1ff91615 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:32.138630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf75f5f2-d49f-445b-b390-4c9f4c054d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:32.139248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f5f2-d49f-445b-b390-4c9f4c054d80 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:32.139580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f5f2-d49f-445b-b390-4c9f4c054d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:32.139806 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf75f5f2-d49f-445b-b390-4c9f4c054d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:32.140441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75f5f2-d49f-445b-b390-4c9f4c054d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:32.142027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf75f5f2-d49f-445b-b390-4c9f4c054d80 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:32.155427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1315/2666] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:32 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:32.156813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf707ed-c35c-41fe-97bd-d9de7fdff978 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:32.167312 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf707ed-c35c-41fe-97bd-d9de7fdff978 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:32.172342 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74742c04-0aab-4e21-b7a3-f5b16a49f30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:32.173003 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74742c04-0aab-4e21-b7a3-f5b16a49f30c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:32.173283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74742c04-0aab-4e21-b7a3-f5b16a49f30c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:32.173559 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74742c04-0aab-4e21-b7a3-f5b16a49f30c admin admin] PATH_INFO: `/v3/projects/d3d9417f5c184f72acf09c2af39a7dff/users/e17b54e8f23b49508a0ef15118610566/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:32.175657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74742c04-0aab-4e21-b7a3-f5b16a49f30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3d9417f5c184f72acf09c2af39a7dff, user_id=e17b54e8f23b49508a0ef15118610566, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:32.178329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74742c04-0aab-4e21-b7a3-f5b16a49f30c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:32.189122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1352/2667] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:08:32 2022] PUT /identity/v3/projects/d3d9417f5c184f72acf09c2af39a7dff/users/e17b54e8f23b49508a0ef15118610566/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:32.200822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d7785-b048-4409-b846-f9d6abb69e99 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:32.201110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d7785-b048-4409-b846-f9d6abb69e99 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:32.201385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42d7785-b048-4409-b846-f9d6abb69e99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:32.210229 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:32.233564 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a42d7785-b048-4409-b846-f9d6abb69e99 None None] MFA Rules not processed for user `e17b54e8f23b49508a0ef15118610566`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:32.234803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a42d7785-b048-4409-b846-f9d6abb69e99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:32.307805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1316/2668] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:32 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:32.320446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74742c04-0aab-4e21-b7a3-f5b16a49f30c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:32.322460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-74742c04-0aab-4e21-b7a3-f5b16a49f30c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:32.345935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:32.391494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74742c04-0aab-4e21-b7a3-f5b16a49f30c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:32.396398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39958b17-5b32-4481-950a-cdf068f978dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:32.397004 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39958b17-5b32-4481-950a-cdf068f978dc service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:32.397283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39958b17-5b32-4481-950a-cdf068f978dc service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:32.397556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39958b17-5b32-4481-950a-cdf068f978dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:32.407671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39958b17-5b32-4481-950a-cdf068f978dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:32.409106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39958b17-5b32-4481-950a-cdf068f978dc service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:32.419211 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1353/2669] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:08:32 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:08:32.586790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d7785-b048-4409-b846-f9d6abb69e99 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:32.653709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42d7785-b048-4409-b846-f9d6abb69e99 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:32.658813 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:32.659440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:32.659730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:32.659997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] PATH_INFO: `/v3/users/c4250e2f4bce47daa703bc1b67f8b11f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:32.661170 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4250e2f4bce47daa703bc1b67f8b11f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:32.663136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:32.716445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4250e2f4bce47daa703bc1b67f8b11f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:32.726115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4250e2f4bce47daa703bc1b67f8b11f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:32.738972 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4250e2f4bce47daa703bc1b67f8b11f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:32.739647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4250e2f4bce47daa703bc1b67f8b11f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:32.746430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4250e2f4bce47daa703bc1b67f8b11f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:32.752249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1317/2670] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:08:32 2022] DELETE /identity/v3/users/c4250e2f4bce47daa703bc1b67f8b11f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:32.756974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4efcd1-f7fe-4387-aa28-55370170c927 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:32.757492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4efcd1-f7fe-4387-aa28-55370170c927 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:32.757591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4efcd1-f7fe-4387-aa28-55370170c927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:32.777319 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-9c4efcd1-f7fe-4387-aa28-55370170c927 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:32.778569 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9c4efcd1-f7fe-4387-aa28-55370170c927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:32.836758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1354/2671] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:32.851424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:32.853098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:32.866118 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:32.916247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eb023d-5ee7-4b6e-83d8-a195711b614d admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:32.920735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b5fda-8fbc-4328-b8de-a3dfac6bece5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:32.921774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5fda-8fbc-4328-b8de-a3dfac6bece5 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:32.921774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5fda-8fbc-4328-b8de-a3dfac6bece5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:32.921949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b5fda-8fbc-4328-b8de-a3dfac6bece5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:32.932441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b5fda-8fbc-4328-b8de-a3dfac6bece5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:32.933930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b5fda-8fbc-4328-b8de-a3dfac6bece5 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:32.944313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1318/2672] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:08:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:08:33.216970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4efcd1-f7fe-4387-aa28-55370170c927 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:33.218536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-9c4efcd1-f7fe-4387-aa28-55370170c927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:33.231570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4efcd1-f7fe-4387-aa28-55370170c927 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:33.237079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:33.237719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:33.237993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:33.238339 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] PATH_INFO: `/v3/projects/e7bedd1544c64b8d9f06c23c64ec7c50` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:33.240098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7bedd1544c64b8d9f06c23c64ec7c50)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:33.242050 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:33.291939 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7bedd1544c64b8d9f06c23c64ec7c50', 'request_id': 'req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.298893 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7bedd1544c64b8d9f06c23c64ec7c50', 'request_id': 'req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.308381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7bedd1544c64b8d9f06c23c64ec7c50', 'request_id': 'req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.323899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] The token cache is being invalidate because project e7bedd1544c64b8d9f06c23c64ec7c50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:08:33.324184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.324809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.326324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1355/2673] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:08:33 2022] DELETE /identity/v3/projects/e7bedd1544c64b8d9f06c23c64ec7c50 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:33.331316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b5fda-8fbc-4328-b8de-a3dfac6bece5 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:33.343152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b5fda-8fbc-4328-b8de-a3dfac6bece5 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:33.408306 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:33.408846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:33.409119 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:33.409389 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] PATH_INFO: `/v3/users/e17b54e8f23b49508a0ef15118610566` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:33.410649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e17b54e8f23b49508a0ef15118610566)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:33.412568 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:33.467669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e17b54e8f23b49508a0ef15118610566'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.476763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e17b54e8f23b49508a0ef15118610566'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.489466 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e17b54e8f23b49508a0ef15118610566'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.490157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e17b54e8f23b49508a0ef15118610566'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.497312 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-490f7f57-3c61-4adc-b717-a20013a55b01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e17b54e8f23b49508a0ef15118610566'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.503807 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1319/2674] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:08:33 2022] DELETE /identity/v3/users/e17b54e8f23b49508a0ef15118610566 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:33.803778 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:33.805265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:33.819264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3d6db-bd5a-4f34-8306-87a00c1f25e2 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:33.869373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:33.883458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:33.884098 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:33.884375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:33.884673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] PATH_INFO: `/v3/projects/d3d9417f5c184f72acf09c2af39a7dff` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:33.885808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3d9417f5c184f72acf09c2af39a7dff)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:33.887751 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:33.937074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3d9417f5c184f72acf09c2af39a7dff', 'request_id': 'req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.943761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3d9417f5c184f72acf09c2af39a7dff', 'request_id': 'req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.949182 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3d9417f5c184f72acf09c2af39a7dff', 'request_id': 'req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.965566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] The token cache is being invalidate because project d3d9417f5c184f72acf09c2af39a7dff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:08:33.965940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.966804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5c5d3a21-3ba4-4b55-99ef-539732b7cd37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:33.969179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1356/2675] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:08:33 2022] DELETE /identity/v3/projects/d3d9417f5c184f72acf09c2af39a7dff => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:33.975705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948f5ad-e852-4114-aab0-fb2133e07723 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:33.976251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948f5ad-e852-4114-aab0-fb2133e07723 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:33.976251 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e948f5ad-e852-4114-aab0-fb2133e07723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:33.996465 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-e948f5ad-e852-4114-aab0-fb2133e07723 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:33.997662 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e948f5ad-e852-4114-aab0-fb2133e07723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:34.023674 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:34.060433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1320/2676] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:34.068553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49533a2-64fd-4ded-9a8d-900ae5d3ac60 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.068839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49533a2-64fd-4ded-9a8d-900ae5d3ac60 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.069113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49533a2-64fd-4ded-9a8d-900ae5d3ac60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.090404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-d49533a2-64fd-4ded-9a8d-900ae5d3ac60 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:34.091612 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d49533a2-64fd-4ded-9a8d-900ae5d3ac60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:34.100806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1357/2677] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:34.105791 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948f5ad-e852-4114-aab0-fb2133e07723 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.118766 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e948f5ad-e852-4114-aab0-fb2133e07723 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.124296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbb310-057a-41cf-97e2-77a452044570 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.124932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbb310-057a-41cf-97e2-77a452044570 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.125205 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbb310-057a-41cf-97e2-77a452044570 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.125498 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dbb310-057a-41cf-97e2-77a452044570 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.126175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbb310-057a-41cf-97e2-77a452044570 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:08:34.126472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbb310-057a-41cf-97e2-77a452044570 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.128070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dbb310-057a-41cf-97e2-77a452044570 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.140074 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1321/2678] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:08:34.146812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49533a2-64fd-4ded-9a8d-900ae5d3ac60 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.156381 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49533a2-64fd-4ded-9a8d-900ae5d3ac60 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.162745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f05b95-a470-4a17-a3f8-6bf3070b05a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.163344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f05b95-a470-4a17-a3f8-6bf3070b05a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.163617 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f05b95-a470-4a17-a3f8-6bf3070b05a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.163882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f05b95-a470-4a17-a3f8-6bf3070b05a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.164519 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f05b95-a470-4a17-a3f8-6bf3070b05a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.166431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f05b95-a470-4a17-a3f8-6bf3070b05a4 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.202837 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1358/2679] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:08:34 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:34.209025 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbb310-057a-41cf-97e2-77a452044570 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.219419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dbb310-057a-41cf-97e2-77a452044570 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.290613 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b38c75-ff56-4628-86e0-41ae934d1a31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.291250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b38c75-ff56-4628-86e0-41ae934d1a31 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.291531 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b38c75-ff56-4628-86e0-41ae934d1a31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.291801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b38c75-ff56-4628-86e0-41ae934d1a31 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.292430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b38c75-ff56-4628-86e0-41ae934d1a31 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.294391 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b38c75-ff56-4628-86e0-41ae934d1a31 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.319616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1322/2680] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:08:34 2022] POST /identity/v3/users => generated 286 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:34.320901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f05b95-a470-4a17-a3f8-6bf3070b05a4 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.331096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f05b95-a470-4a17-a3f8-6bf3070b05a4 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.336601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6922f-6473-4900-b519-71beb90822e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.337218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6922f-6473-4900-b519-71beb90822e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.337497 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6922f-6473-4900-b519-71beb90822e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.337754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e6922f-6473-4900-b519-71beb90822e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.338428 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6922f-6473-4900-b519-71beb90822e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.340034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e6922f-6473-4900-b519-71beb90822e6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.351489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1359/2681] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:34.355522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b38c75-ff56-4628-86e0-41ae934d1a31 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.366390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b38c75-ff56-4628-86e0-41ae934d1a31 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.371136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ea4171-b3f1-4b02-810e-b733f4491f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.371775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4171-b3f1-4b02-810e-b733f4491f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.372068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4171-b3f1-4b02-810e-b733f4491f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.372354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ea4171-b3f1-4b02-810e-b733f4491f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.373005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea4171-b3f1-4b02-810e-b733f4491f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.375140 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ea4171-b3f1-4b02-810e-b733f4491f15 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.386559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1323/2682] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:34.392240 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6922f-6473-4900-b519-71beb90822e6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.403134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e6922f-6473-4900-b519-71beb90822e6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.407993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651a6c9-104b-48bf-a868-2a798d76bc4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.408647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651a6c9-104b-48bf-a868-2a798d76bc4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.408925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651a6c9-104b-48bf-a868-2a798d76bc4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.409199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9651a6c9-104b-48bf-a868-2a798d76bc4b admin admin] PATH_INFO: `/v3/projects/38190eb9a5f247c09c0b4f5e1c6fe1cd/users/0a5d66e9d7244bceab04ee8eb98a9893/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.423877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651a6c9-104b-48bf-a868-2a798d76bc4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=38190eb9a5f247c09c0b4f5e1c6fe1cd, user_id=0a5d66e9d7244bceab04ee8eb98a9893, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.426675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9651a6c9-104b-48bf-a868-2a798d76bc4b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.441718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1360/2683] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:08:34 2022] PUT /identity/v3/projects/38190eb9a5f247c09c0b4f5e1c6fe1cd/users/0a5d66e9d7244bceab04ee8eb98a9893/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:08:34.456493 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08401f-e004-4b87-b865-accb84a04b89 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.456715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08401f-e004-4b87-b865-accb84a04b89 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.456994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab08401f-e004-4b87-b865-accb84a04b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.486218 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ab08401f-e004-4b87-b865-accb84a04b89 None None] MFA Rules not processed for user `0a5d66e9d7244bceab04ee8eb98a9893`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:34.487442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ab08401f-e004-4b87-b865-accb84a04b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:34.566801 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1324/2684] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:34 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:08:34.571844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651a6c9-104b-48bf-a868-2a798d76bc4b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.637806 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9651a6c9-104b-48bf-a868-2a798d76bc4b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.643441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b3506-4486-4678-bb38-317221201f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.644057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b3506-4486-4678-bb38-317221201f60 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.644334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b3506-4486-4678-bb38-317221201f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.644604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324b3506-4486-4678-bb38-317221201f60 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.645237 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b3506-4486-4678-bb38-317221201f60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.647199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324b3506-4486-4678-bb38-317221201f60 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.671655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:34.681114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1361/2685] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:08:34 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:34.685443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08401f-e004-4b87-b865-accb84a04b89 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.696573 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab08401f-e004-4b87-b865-accb84a04b89 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.721160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:34.755315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771dd22-4d58-4adf-8f98-f02284a5592b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.755979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771dd22-4d58-4adf-8f98-f02284a5592b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.756255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771dd22-4d58-4adf-8f98-f02284a5592b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.756521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4771dd22-4d58-4adf-8f98-f02284a5592b admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.757155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771dd22-4d58-4adf-8f98-f02284a5592b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.759106 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4771dd22-4d58-4adf-8f98-f02284a5592b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.786284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1325/2686] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:08:34 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:34.790584 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b3506-4486-4678-bb38-317221201f60 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.801016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324b3506-4486-4678-bb38-317221201f60 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.806864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4ee48-3ec1-4ccf-9dd5-0bf543834f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.807493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4ee48-3ec1-4ccf-9dd5-0bf543834f64 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.807846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4ee48-3ec1-4ccf-9dd5-0bf543834f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.808116 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb4ee48-3ec1-4ccf-9dd5-0bf543834f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.808749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4ee48-3ec1-4ccf-9dd5-0bf543834f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.810373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb4ee48-3ec1-4ccf-9dd5-0bf543834f64 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.821203 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1362/2687] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:08:34.825275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771dd22-4d58-4adf-8f98-f02284a5592b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.835542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4771dd22-4d58-4adf-8f98-f02284a5592b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.840536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95375a12-2884-4e63-b7e7-6a155a1f4d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.841193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95375a12-2884-4e63-b7e7-6a155a1f4d2c admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.841470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95375a12-2884-4e63-b7e7-6a155a1f4d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.841742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95375a12-2884-4e63-b7e7-6a155a1f4d2c admin admin] PATH_INFO: `/v3/projects/5e38f913e88643bca47711ed8ff8a430/users/0b7d2eaddde94986890c2d10b1cb3e1c/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.857882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95375a12-2884-4e63-b7e7-6a155a1f4d2c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e38f913e88643bca47711ed8ff8a430, user_id=0b7d2eaddde94986890c2d10b1cb3e1c, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.860643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95375a12-2884-4e63-b7e7-6a155a1f4d2c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.875941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1326/2688] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:08:34 2022] PUT /identity/v3/projects/5e38f913e88643bca47711ed8ff8a430/users/0b7d2eaddde94986890c2d10b1cb3e1c/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:34.880184 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4ee48-3ec1-4ccf-9dd5-0bf543834f64 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.890603 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb4ee48-3ec1-4ccf-9dd5-0bf543834f64 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.947683 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0f040d-9674-4b40-9056-fffbf6b056a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.948316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f040d-9674-4b40-9056-fffbf6b056a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.948596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f040d-9674-4b40-9056-fffbf6b056a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.948867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f040d-9674-4b40-9056-fffbf6b056a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.949536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f040d-9674-4b40-9056-fffbf6b056a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.951248 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0f040d-9674-4b40-9056-fffbf6b056a8 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.961668 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1363/2689] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:34.966105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95375a12-2884-4e63-b7e7-6a155a1f4d2c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:34.976741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95375a12-2884-4e63-b7e7-6a155a1f4d2c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:34.982225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212af95-ca54-469d-8238-6a182cdbcfb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:34.982888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212af95-ca54-469d-8238-6a182cdbcfb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:34.983175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212af95-ca54-469d-8238-6a182cdbcfb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:34.983445 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1212af95-ca54-469d-8238-6a182cdbcfb0 admin admin] PATH_INFO: `/v3/projects/5e38f913e88643bca47711ed8ff8a430/users/0b7d2eaddde94986890c2d10b1cb3e1c/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:34.985358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212af95-ca54-469d-8238-6a182cdbcfb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e38f913e88643bca47711ed8ff8a430, user_id=0b7d2eaddde94986890c2d10b1cb3e1c, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:34.988382 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1212af95-ca54-469d-8238-6a182cdbcfb0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:34.999300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1327/2690] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:08:34 2022] PUT /identity/v3/projects/5e38f913e88643bca47711ed8ff8a430/users/0b7d2eaddde94986890c2d10b1cb3e1c/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:35.010927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a654-e5af-40a5-b198-9ca124896129 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:35.011212 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a654-e5af-40a5-b198-9ca124896129 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:35.011484 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3840a654-e5af-40a5-b198-9ca124896129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:35.040653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3840a654-e5af-40a5-b198-9ca124896129 None None] MFA Rules not processed for user `0b7d2eaddde94986890c2d10b1cb3e1c`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:35.041926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3840a654-e5af-40a5-b198-9ca124896129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:35.117378 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1364/2691] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:35 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:35.129008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212af95-ca54-469d-8238-6a182cdbcfb0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:35.131446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1212af95-ca54-469d-8238-6a182cdbcfb0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:35.198553 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1212af95-ca54-469d-8238-6a182cdbcfb0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:35.202831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55a8d5-6b4d-4f30-8434-be7c118bbf2f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:35.203433 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55a8d5-6b4d-4f30-8434-be7c118bbf2f service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:35.203713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55a8d5-6b4d-4f30-8434-be7c118bbf2f service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:35.203983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c55a8d5-6b4d-4f30-8434-be7c118bbf2f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:35.214440 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55a8d5-6b4d-4f30-8434-be7c118bbf2f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:35.215889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c55a8d5-6b4d-4f30-8434-be7c118bbf2f service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:35.225966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1328/2692] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:08:35 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:08:35.323703 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a654-e5af-40a5-b198-9ca124896129 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:35.325218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3840a654-e5af-40a5-b198-9ca124896129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:35.337530 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3840a654-e5af-40a5-b198-9ca124896129 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:35.341589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55cd9037-549d-4a89-8a47-0a0edfa3e92c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:35.342193 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9037-549d-4a89-8a47-0a0edfa3e92c service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:35.342470 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9037-549d-4a89-8a47-0a0edfa3e92c service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:35.342732 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55cd9037-549d-4a89-8a47-0a0edfa3e92c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:35.400820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd9037-549d-4a89-8a47-0a0edfa3e92c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:35.402303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55cd9037-549d-4a89-8a47-0a0edfa3e92c service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:35.411249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1365/2693] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:08:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:08:45.221157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=58,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:45.407453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:49.121240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55a8d5-6b4d-4f30-8434-be7c118bbf2f service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:49.122947 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0c55a8d5-6b4d-4f30-8434-be7c118bbf2f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:49.192270 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c55a8d5-6b4d-4f30-8434-be7c118bbf2f service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:49.197987 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:49.198378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:49.198635 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:49.198940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] PATH_INFO: `/v3/users/8c7aca22245745c49b13d1265c920e96` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:49.200188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c7aca22245745c49b13d1265c920e96)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:49.202110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:49.254620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c7aca22245745c49b13d1265c920e96'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.264218 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c7aca22245745c49b13d1265c920e96'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.278630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c7aca22245745c49b13d1265c920e96'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.279417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c7aca22245745c49b13d1265c920e96'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.287786 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c7aca22245745c49b13d1265c920e96'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.294527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1329/2694] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:08:49 2022] DELETE /identity/v3/users/8c7aca22245745c49b13d1265c920e96 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:49.299094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc445c-5f4b-425c-b25c-b2197f40d9af None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:49.299900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc445c-5f4b-425c-b25c-b2197f40d9af None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:49.300364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfc445c-5f4b-425c-b25c-b2197f40d9af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:49.328725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-6dfc445c-5f4b-425c-b25c-b2197f40d9af None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:49.330590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6dfc445c-5f4b-425c-b25c-b2197f40d9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:49.396957 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1366/2695] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:49.408581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:49.410183 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:49.479643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbaf7a4-0a6a-48c2-8cee-b27eb535e319 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:49.483698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79333eb4-6cb8-4e7d-9c24-872d16606749 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:49.484331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79333eb4-6cb8-4e7d-9c24-872d16606749 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:49.484615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79333eb4-6cb8-4e7d-9c24-872d16606749 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:49.484909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79333eb4-6cb8-4e7d-9c24-872d16606749 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:49.494369 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79333eb4-6cb8-4e7d-9c24-872d16606749 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:49.495709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79333eb4-6cb8-4e7d-9c24-872d16606749 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:49.506936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1330/2696] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:08:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:08:49.787926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc445c-5f4b-425c-b25c-b2197f40d9af None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:49.789537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-6dfc445c-5f4b-425c-b25c-b2197f40d9af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:49.805278 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfc445c-5f4b-425c-b25c-b2197f40d9af None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:49.812191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:49.812812 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:49.813084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:49.813349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] PATH_INFO: `/v3/projects/f4f30c8da851457f96e1f2a309a14769` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:49.814714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f30c8da851457f96e1f2a309a14769)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:49.817099 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:49.867223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f30c8da851457f96e1f2a309a14769', 'request_id': 'req-38d0dc55-1dc6-4232-bac8-de86c337598f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.873756 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f30c8da851457f96e1f2a309a14769', 'request_id': 'req-38d0dc55-1dc6-4232-bac8-de86c337598f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.879927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f30c8da851457f96e1f2a309a14769', 'request_id': 'req-38d0dc55-1dc6-4232-bac8-de86c337598f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.896102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] The token cache is being invalidate because project f4f30c8da851457f96e1f2a309a14769 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:08:49.896389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.897006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-38d0dc55-1dc6-4232-bac8-de86c337598f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:08:49.898511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1367/2697] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:08:49 2022] DELETE /identity/v3/projects/f4f30c8da851457f96e1f2a309a14769 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:49.904310 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0fb8f-38c1-4c7e-8e92-02171a186326 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:49.904597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0fb8f-38c1-4c7e-8e92-02171a186326 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:49.904901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0fb8f-38c1-4c7e-8e92-02171a186326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:49.925348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c2e0fb8f-38c1-4c7e-8e92-02171a186326 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:49.926612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c2e0fb8f-38c1-4c7e-8e92-02171a186326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:49.945990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:49.994512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1331/2698] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:50.002532 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2c19a-a72d-457c-8214-c5379444a50f None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:50.002818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2c19a-a72d-457c-8214-c5379444a50f None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:50.003091 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2c19a-a72d-457c-8214-c5379444a50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:50.025949 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-5eb2c19a-a72d-457c-8214-c5379444a50f None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:50.027248 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5eb2c19a-a72d-457c-8214-c5379444a50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:50.036741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1368/2699] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:50.041591 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0fb8f-38c1-4c7e-8e92-02171a186326 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:50.052155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0fb8f-38c1-4c7e-8e92-02171a186326 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:50.058257 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c87645-8441-4d76-9e8b-21b068526179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:50.058875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c87645-8441-4d76-9e8b-21b068526179 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:50.059160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c87645-8441-4d76-9e8b-21b068526179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:50.059434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c87645-8441-4d76-9e8b-21b068526179 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:50.060159 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c87645-8441-4d76-9e8b-21b068526179 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:08:50.060466 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c87645-8441-4d76-9e8b-21b068526179 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:50.062152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c87645-8441-4d76-9e8b-21b068526179 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:50.075128 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1332/2700] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:08:50.080435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2c19a-a72d-457c-8214-c5379444a50f None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:50.090630 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2c19a-a72d-457c-8214-c5379444a50f None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:50.095622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fe322-b682-426d-a5c8-470dbc8480ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:50.096227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fe322-b682-426d-a5c8-470dbc8480ba admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:50.096503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fe322-b682-426d-a5c8-470dbc8480ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:50.096773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431fe322-b682-426d-a5c8-470dbc8480ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:50.097398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fe322-b682-426d-a5c8-470dbc8480ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:50.099326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431fe322-b682-426d-a5c8-470dbc8480ba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:50.138596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1369/2701] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:08:50 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:50.142995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c87645-8441-4d76-9e8b-21b068526179 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:50.153506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c87645-8441-4d76-9e8b-21b068526179 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:50.221583 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f24bf7-ffec-4e54-822b-2a80de33beec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:50.222210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f24bf7-ffec-4e54-822b-2a80de33beec admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:50.222524 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f24bf7-ffec-4e54-822b-2a80de33beec admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:50.222797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f24bf7-ffec-4e54-822b-2a80de33beec admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:50.223393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f24bf7-ffec-4e54-822b-2a80de33beec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:50.225409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f24bf7-ffec-4e54-822b-2a80de33beec admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:50.252569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1333/2702] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:08:50 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:08:50.258917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fe322-b682-426d-a5c8-470dbc8480ba admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:50.269971 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431fe322-b682-426d-a5c8-470dbc8480ba admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:50.275391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb7c5e-ba71-4788-82fb-fc34d5a02129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:50.276029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb7c5e-ba71-4788-82fb-fc34d5a02129 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:50.276435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb7c5e-ba71-4788-82fb-fc34d5a02129 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:50.276726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cb7c5e-ba71-4788-82fb-fc34d5a02129 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:50.277365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb7c5e-ba71-4788-82fb-fc34d5a02129 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:50.279020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cb7c5e-ba71-4788-82fb-fc34d5a02129 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:50.293027 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1370/2703] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:50.297228 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f24bf7-ffec-4e54-822b-2a80de33beec admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:50.308494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f24bf7-ffec-4e54-822b-2a80de33beec admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:50.313292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2240abc5-f80b-4c93-9df4-3feb0652fee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:50.313905 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240abc5-f80b-4c93-9df4-3feb0652fee7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:50.314222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240abc5-f80b-4c93-9df4-3feb0652fee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:50.314499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2240abc5-f80b-4c93-9df4-3feb0652fee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:50.315148 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240abc5-f80b-4c93-9df4-3feb0652fee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:50.316742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2240abc5-f80b-4c93-9df4-3feb0652fee7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:50.327907 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1334/2704] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:08:50.332338 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb7c5e-ba71-4788-82fb-fc34d5a02129 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:50.342988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cb7c5e-ba71-4788-82fb-fc34d5a02129 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:50.348129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f27404-ab57-4ea0-bf22-5cd7d51e7547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:50.348845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f27404-ab57-4ea0-bf22-5cd7d51e7547 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:50.349227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f27404-ab57-4ea0-bf22-5cd7d51e7547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:50.349377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f27404-ab57-4ea0-bf22-5cd7d51e7547 admin admin] PATH_INFO: `/v3/projects/5e15dced44474a3c9d3072e3682155cc/users/5619c911077e4c50b56022498aaa1749/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:50.366360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f27404-ab57-4ea0-bf22-5cd7d51e7547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e15dced44474a3c9d3072e3682155cc, user_id=5619c911077e4c50b56022498aaa1749, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:50.368974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f27404-ab57-4ea0-bf22-5cd7d51e7547 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:50.383460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1371/2705] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:08:50 2022] PUT /identity/v3/projects/5e15dced44474a3c9d3072e3682155cc/users/5619c911077e4c50b56022498aaa1749/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:08:50.394530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7bfd2-a5ce-40d5-926a-a460fbc3b365 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:50.394814 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7bfd2-a5ce-40d5-926a-a460fbc3b365 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:50.395092 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7bfd2-a5ce-40d5-926a-a460fbc3b365 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:50.425895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-94e7bfd2-a5ce-40d5-926a-a460fbc3b365 None None] MFA Rules not processed for user `5619c911077e4c50b56022498aaa1749`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:08:50.427135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-94e7bfd2-a5ce-40d5-926a-a460fbc3b365 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:50.497865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1335/2706] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:08:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:08:50.510802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f27404-ab57-4ea0-bf22-5cd7d51e7547 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:08:50.513010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f4f27404-ab57-4ea0-bf22-5cd7d51e7547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:08:50.537985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:08:50.577303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f27404-ab57-4ea0-bf22-5cd7d51e7547 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:08:50.580876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bbb1e0-e0e7-4698-9c12-1403b351927b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:08:50.581488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbb1e0-e0e7-4698-9c12-1403b351927b service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:08:50.581767 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbb1e0-e0e7-4698-9c12-1403b351927b service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:08:50.582031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bbb1e0-e0e7-4698-9c12-1403b351927b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:08:50.590926 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbb1e0-e0e7-4698-9c12-1403b351927b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:08:50.592467 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bbb1e0-e0e7-4698-9c12-1403b351927b service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:08:50.602228 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1372/2707] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:08:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:00.495110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:00.597666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=17 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:06.388245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7bfd2-a5ce-40d5-926a-a460fbc3b365 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:06.498988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7bfd2-a5ce-40d5-926a-a460fbc3b365 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:06.503746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:06.504463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:06.504736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:06.505009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] PATH_INFO: `/v3/users/5619c911077e4c50b56022498aaa1749` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:06.506234 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5619c911077e4c50b56022498aaa1749)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:06.508175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:06.560386 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5619c911077e4c50b56022498aaa1749'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:06.569592 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5619c911077e4c50b56022498aaa1749'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:06.583887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5619c911077e4c50b56022498aaa1749'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:06.584746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5619c911077e4c50b56022498aaa1749'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:06.592920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5619c911077e4c50b56022498aaa1749'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:06.599807 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1336/2708] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:06 2022] DELETE /identity/v3/users/5619c911077e4c50b56022498aaa1749 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:06.604404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3e013-08b9-4b31-92be-617bacf96c07 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:06.604903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3e013-08b9-4b31-92be-617bacf96c07 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:06.605008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c3e013-08b9-4b31-92be-617bacf96c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:06.655476 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-03c3e013-08b9-4b31-92be-617bacf96c07 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:06.656599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-03c3e013-08b9-4b31-92be-617bacf96c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:06.717112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1373/2709] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:06.727492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:06.729155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:06.772876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:06.809276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38233f37-8b93-4b0b-a180-684920314fc0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:06.813052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b506014-5e25-491b-b3c6-ae8ddb5da805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:06.813676 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b506014-5e25-491b-b3c6-ae8ddb5da805 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:06.813955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b506014-5e25-491b-b3c6-ae8ddb5da805 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:06.814273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b506014-5e25-491b-b3c6-ae8ddb5da805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:06.824612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b506014-5e25-491b-b3c6-ae8ddb5da805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:06.826280 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b506014-5e25-491b-b3c6-ae8ddb5da805 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:06.838484 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1337/2710] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:09:07.402722 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3e013-08b9-4b31-92be-617bacf96c07 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:07.404169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-03c3e013-08b9-4b31-92be-617bacf96c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:07.418425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c3e013-08b9-4b31-92be-617bacf96c07 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:07.423048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:07.424316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:07.424609 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:07.424922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] PATH_INFO: `/v3/projects/5e15dced44474a3c9d3072e3682155cc` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:07.426352 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e15dced44474a3c9d3072e3682155cc)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:07.428384 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:07.480257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e15dced44474a3c9d3072e3682155cc', 'request_id': 'req-6385e4d0-4621-48d5-b657-b3acb53692ba'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:07.486300 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e15dced44474a3c9d3072e3682155cc', 'request_id': 'req-6385e4d0-4621-48d5-b657-b3acb53692ba'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:07.492460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e15dced44474a3c9d3072e3682155cc', 'request_id': 'req-6385e4d0-4621-48d5-b657-b3acb53692ba'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:07.506548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] The token cache is being invalidate because project 5e15dced44474a3c9d3072e3682155cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:07.506834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:07.507632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-6385e4d0-4621-48d5-b657-b3acb53692ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:07.509282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1374/2711] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:07 2022] DELETE /identity/v3/projects/5e15dced44474a3c9d3072e3682155cc => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:07.514348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806838d-188f-4813-9088-dca1b508a5d2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:07.514844 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806838d-188f-4813-9088-dca1b508a5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:07.514951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806838d-188f-4813-9088-dca1b508a5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:07.566319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-5806838d-188f-4813-9088-dca1b508a5d2 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:07.567630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-5806838d-188f-4813-9088-dca1b508a5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:07.638769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1338/2712] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:07.647111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a00ea6-5317-41b9-bee6-e6d8411a24b6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:07.647664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a00ea6-5317-41b9-bee6-e6d8411a24b6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:07.647664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a00ea6-5317-41b9-bee6-e6d8411a24b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:07.672509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-e8a00ea6-5317-41b9-bee6-e6d8411a24b6 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:07.674289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-e8a00ea6-5317-41b9-bee6-e6d8411a24b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:07.686993 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1375/2713] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:07.692506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806838d-188f-4813-9088-dca1b508a5d2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:07.702432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806838d-188f-4813-9088-dca1b508a5d2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:07.707996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0ae74-4a20-48f5-a914-f3598434ca70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:07.708606 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0ae74-4a20-48f5-a914-f3598434ca70 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:07.708885 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0ae74-4a20-48f5-a914-f3598434ca70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:07.709154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb0ae74-4a20-48f5-a914-f3598434ca70 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:07.709798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0ae74-4a20-48f5-a914-f3598434ca70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:09:07.710072 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0ae74-4a20-48f5-a914-f3598434ca70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:07.711696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb0ae74-4a20-48f5-a914-f3598434ca70 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:07.725102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1339/2714] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:09:07.729789 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a00ea6-5317-41b9-bee6-e6d8411a24b6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:07.739870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a00ea6-5317-41b9-bee6-e6d8411a24b6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:07.745453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5d2a-9dc7-4332-b320-d94c2137166e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:07.746056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5d2a-9dc7-4332-b320-d94c2137166e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:07.746372 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5d2a-9dc7-4332-b320-d94c2137166e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:07.746647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-787a5d2a-9dc7-4332-b320-d94c2137166e admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:07.747294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a5d2a-9dc7-4332-b320-d94c2137166e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:07.749257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-787a5d2a-9dc7-4332-b320-d94c2137166e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:07.777636 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1376/2715] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:09:07 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:07.782822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0ae74-4a20-48f5-a914-f3598434ca70 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:07.793964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb0ae74-4a20-48f5-a914-f3598434ca70 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:07.855195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2eaa5b-5e15-425e-859c-e8dc1c14095a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:07.855772 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2eaa5b-5e15-425e-859c-e8dc1c14095a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:07.856053 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2eaa5b-5e15-425e-859c-e8dc1c14095a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:07.856364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2eaa5b-5e15-425e-859c-e8dc1c14095a admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:07.857033 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2eaa5b-5e15-425e-859c-e8dc1c14095a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:07.859016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2eaa5b-5e15-425e-859c-e8dc1c14095a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:07.883202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1340/2716] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:09:07 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:07.887634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5d2a-9dc7-4332-b320-d94c2137166e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:07.899481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-787a5d2a-9dc7-4332-b320-d94c2137166e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:07.904457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99a37d-e54a-49ea-b3a9-d7a155af9119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:07.905059 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99a37d-e54a-49ea-b3a9-d7a155af9119 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:07.905334 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99a37d-e54a-49ea-b3a9-d7a155af9119 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:07.905647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd99a37d-e54a-49ea-b3a9-d7a155af9119 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:07.906329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99a37d-e54a-49ea-b3a9-d7a155af9119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:07.907981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd99a37d-e54a-49ea-b3a9-d7a155af9119 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:07.918628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1377/2717] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:07.923378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2eaa5b-5e15-425e-859c-e8dc1c14095a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:07.934388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2eaa5b-5e15-425e-859c-e8dc1c14095a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:07.939580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e774041-adf7-42a4-ad35-cb7dcfa8679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:07.940187 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e774041-adf7-42a4-ad35-cb7dcfa8679e admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:07.940469 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e774041-adf7-42a4-ad35-cb7dcfa8679e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:07.940734 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e774041-adf7-42a4-ad35-cb7dcfa8679e admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:07.941478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e774041-adf7-42a4-ad35-cb7dcfa8679e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:07.943636 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e774041-adf7-42a4-ad35-cb7dcfa8679e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:07.954541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1341/2718] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:07.959025 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99a37d-e54a-49ea-b3a9-d7a155af9119 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:07.969824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd99a37d-e54a-49ea-b3a9-d7a155af9119 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:07.975798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b60c67-cb27-4230-a049-0a6339dffa30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:07.976462 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b60c67-cb27-4230-a049-0a6339dffa30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:07.976740 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b60c67-cb27-4230-a049-0a6339dffa30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:07.977007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b60c67-cb27-4230-a049-0a6339dffa30 admin admin] PATH_INFO: `/v3/projects/ec2bf33b461f441380f0f820d9dc2d31/users/1c16c6ef8acc4e8aabe733ea0ed378a2/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:07.993158 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b60c67-cb27-4230-a049-0a6339dffa30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec2bf33b461f441380f0f820d9dc2d31, user_id=1c16c6ef8acc4e8aabe733ea0ed378a2, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:07.995838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b60c67-cb27-4230-a049-0a6339dffa30 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:07.997975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:08.012805 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1378/2719] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:07 2022] PUT /identity/v3/projects/ec2bf33b461f441380f0f820d9dc2d31/users/1c16c6ef8acc4e8aabe733ea0ed378a2/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:08.023288 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc13a4-cc7c-4c14-8927-2d1e9433ddb6 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:08.023599 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc13a4-cc7c-4c14-8927-2d1e9433ddb6 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:08.023872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc13a4-cc7c-4c14-8927-2d1e9433ddb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:08.038316 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:08.057166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-a2bc13a4-cc7c-4c14-8927-2d1e9433ddb6 None None] MFA Rules not processed for user `1c16c6ef8acc4e8aabe733ea0ed378a2`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:08.058414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a2bc13a4-cc7c-4c14-8927-2d1e9433ddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:08.124938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1342/2720] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:08 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:08.140263 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b60c67-cb27-4230-a049-0a6339dffa30 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:08.142122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c9b60c67-cb27-4230-a049-0a6339dffa30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:08.200892 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b60c67-cb27-4230-a049-0a6339dffa30 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:08.204983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2220486-2010-440f-abfa-603c1a225ee6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:08.205581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2220486-2010-440f-abfa-603c1a225ee6 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:08.205851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2220486-2010-440f-abfa-603c1a225ee6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:08.206118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2220486-2010-440f-abfa-603c1a225ee6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:08.214986 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2220486-2010-440f-abfa-603c1a225ee6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:08.216411 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2220486-2010-440f-abfa-603c1a225ee6 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:08.225218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1379/2721] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:08 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:11.654388 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc13a4-cc7c-4c14-8927-2d1e9433ddb6 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:11.656288 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-a2bc13a4-cc7c-4c14-8927-2d1e9433ddb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:11.730901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc13a4-cc7c-4c14-8927-2d1e9433ddb6 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:11.737790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:11.738648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:11.738988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:11.739449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:11.741985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:11.810898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:11.812276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:11.822043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1343/2722] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:11 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:17.530129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2220486-2010-440f-abfa-603c1a225ee6 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:17.599077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2220486-2010-440f-abfa-603c1a225ee6 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:17.604531 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:17.605164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:17.605439 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:17.605709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] PATH_INFO: `/v3/users/1c16c6ef8acc4e8aabe733ea0ed378a2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:17.606964 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c16c6ef8acc4e8aabe733ea0ed378a2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:17.608903 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:17.622181 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:17.624136 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:17.672242 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c16c6ef8acc4e8aabe733ea0ed378a2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:17.673431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c16c6ef8acc4e8aabe733ea0ed378a2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:17.686069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c16c6ef8acc4e8aabe733ea0ed378a2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:17.698798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=100 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:17.699570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c16c6ef8acc4e8aabe733ea0ed378a2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:17.700346 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-48064813-5b7c-402d-b18c-b3229a60a61f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c16c6ef8acc4e8aabe733ea0ed378a2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:17.714205 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1380/2723] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:17 2022] DELETE /identity/v3/users/1c16c6ef8acc4e8aabe733ea0ed378a2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:17.718678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01c44a-a6d6-4e22-8af7-d266c20c415d None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:17.719045 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01c44a-a6d6-4e22-8af7-d266c20c415d None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:17.719394 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b01c44a-a6d6-4e22-8af7-d266c20c415d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:17.737021 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95288e38-8786-4833-814c-9d63349016e3 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:17.739016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3b01c44a-a6d6-4e22-8af7-d266c20c415d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:17.740235 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3b01c44a-a6d6-4e22-8af7-d266c20c415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:17.754521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:INSERT=2,keystone:SELECT=88,keystone:DELETE=10 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:17.802274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:17.802928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:17.803275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:17.803562 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:17.805343 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:17.807574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1381/2724] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:09:17.821394 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:17.823145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:17.823553 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01c44a-a6d6-4e22-8af7-d266c20c415d None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:17.825391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3b01c44a-a6d6-4e22-8af7-d266c20c415d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:17.837120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1344/2725] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:17.901686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b01c44a-a6d6-4e22-8af7-d266c20c415d None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:17.905262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae210061-55d3-49b8-9ca1-f87b9490112e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:17.905870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae210061-55d3-49b8-9ca1-f87b9490112e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:17.906178 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae210061-55d3-49b8-9ca1-f87b9490112e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:17.906448 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae210061-55d3-49b8-9ca1-f87b9490112e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:17.916016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae210061-55d3-49b8-9ca1-f87b9490112e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:17.917741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae210061-55d3-49b8-9ca1-f87b9490112e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:17.927664 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1382/2726] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:18.278429 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:18.279886 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:18.315418 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344b545a-ca90-410d-9445-1ab838a456cc service placement] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:18.328541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:18.329334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:18.329716 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:18.330292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] PATH_INFO: `/v3/projects/ec2bf33b461f441380f0f820d9dc2d31` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:18.331565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec2bf33b461f441380f0f820d9dc2d31)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:18.334314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:18.418584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec2bf33b461f441380f0f820d9dc2d31', 'request_id': 'req-8786bb92-c314-4860-84b8-146a6fdb02b4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:18.426047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec2bf33b461f441380f0f820d9dc2d31', 'request_id': 'req-8786bb92-c314-4860-84b8-146a6fdb02b4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:18.433634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec2bf33b461f441380f0f820d9dc2d31', 'request_id': 'req-8786bb92-c314-4860-84b8-146a6fdb02b4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:18.448491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] The token cache is being invalidate because project ec2bf33b461f441380f0f820d9dc2d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:18.448694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:18.449496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8786bb92-c314-4860-84b8-146a6fdb02b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:18.451460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1345/2727] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:18 2022] DELETE /identity/v3/projects/ec2bf33b461f441380f0f820d9dc2d31 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:20.082419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae210061-55d3-49b8-9ca1-f87b9490112e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:20.083800 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ae210061-55d3-49b8-9ca1-f87b9490112e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:20.104757 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae210061-55d3-49b8-9ca1-f87b9490112e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:20.197741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:20.198420 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:20.198702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:20.198979 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] PATH_INFO: `/v3/users/0a5d66e9d7244bceab04ee8eb98a9893` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:20.200142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a5d66e9d7244bceab04ee8eb98a9893)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:20.202096 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:20.257048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a5d66e9d7244bceab04ee8eb98a9893'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.257698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a5d66e9d7244bceab04ee8eb98a9893'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.267830 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a5d66e9d7244bceab04ee8eb98a9893'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.276303 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:20.295874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a5d66e9d7244bceab04ee8eb98a9893'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.297067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a5d66e9d7244bceab04ee8eb98a9893'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.311008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1383/2728] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:20 2022] DELETE /identity/v3/users/0a5d66e9d7244bceab04ee8eb98a9893 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:20.315502 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04defec-95ab-40e7-94b0-010d008d25dd None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:20.315990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04defec-95ab-40e7-94b0-010d008d25dd None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:20.316115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04defec-95ab-40e7-94b0-010d008d25dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:20.338142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f04defec-95ab-40e7-94b0-010d008d25dd None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:20.339044 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f04defec-95ab-40e7-94b0-010d008d25dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:20.382990 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:20.403633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1346/2729] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:20.413611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:20.415014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:20.473741 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045e647-5ceb-4643-88ac-ccce1bd7236c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:20.477558 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f34f28-8adf-4fc5-ac3a-26e4cf2f5496 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:20.478194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f34f28-8adf-4fc5-ac3a-26e4cf2f5496 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:20.478455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f34f28-8adf-4fc5-ac3a-26e4cf2f5496 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:20.478723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f34f28-8adf-4fc5-ac3a-26e4cf2f5496 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:20.488170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f34f28-8adf-4fc5-ac3a-26e4cf2f5496 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:20.489624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f34f28-8adf-4fc5-ac3a-26e4cf2f5496 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:20.499368 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1384/2730] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:09:20.788792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04defec-95ab-40e7-94b0-010d008d25dd None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:20.790649 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f04defec-95ab-40e7-94b0-010d008d25dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:20.808669 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04defec-95ab-40e7-94b0-010d008d25dd None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:20.814210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:20.814985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:20.815474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:20.815757 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] PATH_INFO: `/v3/projects/38190eb9a5f247c09c0b4f5e1c6fe1cd` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:20.816958 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38190eb9a5f247c09c0b4f5e1c6fe1cd)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:20.819178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:20.819336 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f34f28-8adf-4fc5-ac3a-26e4cf2f5496 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:20.820682 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-47f34f28-8adf-4fc5-ac3a-26e4cf2f5496 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:20.835259 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f34f28-8adf-4fc5-ac3a-26e4cf2f5496 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:20.839864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b62eb-d7f8-4d6c-b0d5-773605560e2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:20.840573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b62eb-d7f8-4d6c-b0d5-773605560e2e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:20.840947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b62eb-d7f8-4d6c-b0d5-773605560e2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:20.841223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327b62eb-d7f8-4d6c-b0d5-773605560e2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:20.843026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-327b62eb-d7f8-4d6c-b0d5-773605560e2e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:20.857988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b62eb-d7f8-4d6c-b0d5-773605560e2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:20.859802 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327b62eb-d7f8-4d6c-b0d5-773605560e2e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:20.870492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1385/2731] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:09:20.871571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38190eb9a5f247c09c0b4f5e1c6fe1cd', 'request_id': 'req-8f2e1c22-60d1-4564-ad6c-c19905afa825'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.877822 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38190eb9a5f247c09c0b4f5e1c6fe1cd', 'request_id': 'req-8f2e1c22-60d1-4564-ad6c-c19905afa825'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.883670 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38190eb9a5f247c09c0b4f5e1c6fe1cd', 'request_id': 'req-8f2e1c22-60d1-4564-ad6c-c19905afa825'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.898698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] The token cache is being invalidate because project 38190eb9a5f247c09c0b4f5e1c6fe1cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:20.898976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.899590 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:20.901175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1347/2732] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:20 2022] DELETE /identity/v3/projects/38190eb9a5f247c09c0b4f5e1c6fe1cd => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:20.905139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b62eb-d7f8-4d6c-b0d5-773605560e2e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:20.915243 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327b62eb-d7f8-4d6c-b0d5-773605560e2e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:20.973236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:20.973882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:20.974195 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:20.974511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] PATH_INFO: `/v3/users/0b7d2eaddde94986890c2d10b1cb3e1c` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:20.976010 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b7d2eaddde94986890c2d10b1cb3e1c)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:20.977936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:21.026666 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b7d2eaddde94986890c2d10b1cb3e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.027332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b7d2eaddde94986890c2d10b1cb3e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.035915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b7d2eaddde94986890c2d10b1cb3e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.047690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:21.049121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b7d2eaddde94986890c2d10b1cb3e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.049970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c982594d-64b6-41d7-ba17-bfc1e56877e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b7d2eaddde94986890c2d10b1cb3e1c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.066498 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1386/2733] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:20 2022] DELETE /identity/v3/users/0b7d2eaddde94986890c2d10b1cb3e1c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:21.346462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:21.349711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:21.372275 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2e1c22-60d1-4564-ad6c-c19905afa825 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:21.432409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:21.433050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:21.433327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:21.433595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] PATH_INFO: `/v3/projects/5e38f913e88643bca47711ed8ff8a430` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:21.434798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e38f913e88643bca47711ed8ff8a430)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:21.436759 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:21.490220 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e38f913e88643bca47711ed8ff8a430', 'request_id': 'req-6ec78780-0d04-49ca-950d-dadf7253fb2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.496874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e38f913e88643bca47711ed8ff8a430', 'request_id': 'req-6ec78780-0d04-49ca-950d-dadf7253fb2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.502614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e38f913e88643bca47711ed8ff8a430', 'request_id': 'req-6ec78780-0d04-49ca-950d-dadf7253fb2f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.513372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:21.517862 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] The token cache is being invalidate because project 5e38f913e88643bca47711ed8ff8a430 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:21.518217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.518901 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6ec78780-0d04-49ca-950d-dadf7253fb2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:21.521334 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1348/2734] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:21 2022] DELETE /identity/v3/projects/5e38f913e88643bca47711ed8ff8a430 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:21.526614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d75bc-a52c-4941-947b-502f95221b77 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:21.527126 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d75bc-a52c-4941-947b-502f95221b77 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:21.527245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94d75bc-a52c-4941-947b-502f95221b77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:21.547567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-a94d75bc-a52c-4941-947b-502f95221b77 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:21.548840 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a94d75bc-a52c-4941-947b-502f95221b77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:21.622675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1387/2735] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:09:21.632249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fad1d-6ee1-4801-8f6d-91b17eed5fff None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:21.632615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fad1d-6ee1-4801-8f6d-91b17eed5fff None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:21.632965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2fad1d-6ee1-4801-8f6d-91b17eed5fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:21.654052 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ca2fad1d-6ee1-4801-8f6d-91b17eed5fff None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:21.655417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca2fad1d-6ee1-4801-8f6d-91b17eed5fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:21.665082 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1349/2736] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:21.669672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d75bc-a52c-4941-947b-502f95221b77 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:21.680142 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94d75bc-a52c-4941-947b-502f95221b77 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:21.685237 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838f92e-10ac-428b-90b8-d2c28c2f6f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:21.685929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838f92e-10ac-428b-90b8-d2c28c2f6f06 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:21.686387 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838f92e-10ac-428b-90b8-d2c28c2f6f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:21.686730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3838f92e-10ac-428b-90b8-d2c28c2f6f06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:21.687455 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838f92e-10ac-428b-90b8-d2c28c2f6f06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:09:21.687826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838f92e-10ac-428b-90b8-d2c28c2f6f06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:21.689500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3838f92e-10ac-428b-90b8-d2c28c2f6f06 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:21.702167 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1388/2737] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:09:21.706909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fad1d-6ee1-4801-8f6d-91b17eed5fff None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:21.716940 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2fad1d-6ee1-4801-8f6d-91b17eed5fff None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:21.722198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba16178-c14e-432f-afd7-5e669993afe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:21.722793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba16178-c14e-432f-afd7-5e669993afe2 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:21.723152 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba16178-c14e-432f-afd7-5e669993afe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:21.723514 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba16178-c14e-432f-afd7-5e669993afe2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:21.724169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba16178-c14e-432f-afd7-5e669993afe2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:21.726167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba16178-c14e-432f-afd7-5e669993afe2 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:21.759352 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1350/2738] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:09:21 2022] POST /identity/v3/projects => generated 357 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:09:21.760714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838f92e-10ac-428b-90b8-d2c28c2f6f06 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:21.771007 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3838f92e-10ac-428b-90b8-d2c28c2f6f06 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:21.833079 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287634-49c7-452e-8c42-8525b5093c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:21.833769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287634-49c7-452e-8c42-8525b5093c36 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:21.834051 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287634-49c7-452e-8c42-8525b5093c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:21.834583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b287634-49c7-452e-8c42-8525b5093c36 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:21.835761 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b287634-49c7-452e-8c42-8525b5093c36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:21.837790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b287634-49c7-452e-8c42-8525b5093c36 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:21.864970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1389/2739] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:09:21 2022] POST /identity/v3/users => generated 279 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:21.868934 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba16178-c14e-432f-afd7-5e669993afe2 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:21.884150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba16178-c14e-432f-afd7-5e669993afe2 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:21.888923 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e4ede-491d-4ac7-b20e-202454104967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:21.889555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e4ede-491d-4ac7-b20e-202454104967 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:21.889832 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e4ede-491d-4ac7-b20e-202454104967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:21.890146 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3e4ede-491d-4ac7-b20e-202454104967 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:21.890787 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e4ede-491d-4ac7-b20e-202454104967 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:21.892413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3e4ede-491d-4ac7-b20e-202454104967 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:21.904816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1351/2740] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:21.908835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287634-49c7-452e-8c42-8525b5093c36 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:21.919758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b287634-49c7-452e-8c42-8525b5093c36 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:21.924698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c58e93f-6d73-429a-b4a8-cde1f7c3b823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:21.925324 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e93f-6d73-429a-b4a8-cde1f7c3b823 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:21.925606 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e93f-6d73-429a-b4a8-cde1f7c3b823 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:21.925881 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c58e93f-6d73-429a-b4a8-cde1f7c3b823 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:21.926628 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58e93f-6d73-429a-b4a8-cde1f7c3b823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:21.928551 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c58e93f-6d73-429a-b4a8-cde1f7c3b823 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:21.941423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1390/2741] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:21.945751 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e4ede-491d-4ac7-b20e-202454104967 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:21.956535 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3e4ede-491d-4ac7-b20e-202454104967 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:21.961419 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a09d-c88c-4585-8a6f-0a62d9196428 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:21.962225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a09d-c88c-4585-8a6f-0a62d9196428 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:21.962493 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a09d-c88c-4585-8a6f-0a62d9196428 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:21.962765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a247a09d-c88c-4585-8a6f-0a62d9196428 admin admin] PATH_INFO: `/v3/projects/0ccb24c2806740f1b2f463da0ae426f4/users/0478bb1f87b74eb8b9765068dfc8efc8/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:21.979755 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247a09d-c88c-4585-8a6f-0a62d9196428 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ccb24c2806740f1b2f463da0ae426f4, user_id=0478bb1f87b74eb8b9765068dfc8efc8, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:21.982417 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a247a09d-c88c-4585-8a6f-0a62d9196428 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:22.000197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1352/2742] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:21 2022] PUT /identity/v3/projects/0ccb24c2806740f1b2f463da0ae426f4/users/0478bb1f87b74eb8b9765068dfc8efc8/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:22.010583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6232f1-94ec-4104-bd1b-d56b86a6b3b7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.010867 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6232f1-94ec-4104-bd1b-d56b86a6b3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.011139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6232f1-94ec-4104-bd1b-d56b86a6b3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:22.039197 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3b6232f1-94ec-4104-bd1b-d56b86a6b3b7 None None] MFA Rules not processed for user `0478bb1f87b74eb8b9765068dfc8efc8`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:22.040423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3b6232f1-94ec-4104-bd1b-d56b86a6b3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:22.083069 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:22.107866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1391/2743] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:22.112355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a09d-c88c-4585-8a6f-0a62d9196428 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:22.173631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a247a09d-c88c-4585-8a6f-0a62d9196428 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:22.178346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700741f-b512-4f0f-b5a7-4ccb89f4ba9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:22.178968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700741f-b512-4f0f-b5a7-4ccb89f4ba9a admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.179250 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700741f-b512-4f0f-b5a7-4ccb89f4ba9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.179525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3700741f-b512-4f0f-b5a7-4ccb89f4ba9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:22.180177 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700741f-b512-4f0f-b5a7-4ccb89f4ba9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:22.182687 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3700741f-b512-4f0f-b5a7-4ccb89f4ba9a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:22.215710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1353/2744] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:09:22 2022] POST /identity/v3/projects => generated 357 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:22.219716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6232f1-94ec-4104-bd1b-d56b86a6b3b7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:22.230823 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6232f1-94ec-4104-bd1b-d56b86a6b3b7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:22.288171 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079bb6f-ce63-478a-8bd4-31f076bb2320 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:22.288779 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079bb6f-ce63-478a-8bd4-31f076bb2320 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.289057 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079bb6f-ce63-478a-8bd4-31f076bb2320 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.289325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079bb6f-ce63-478a-8bd4-31f076bb2320 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:22.289969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079bb6f-ce63-478a-8bd4-31f076bb2320 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:22.291936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079bb6f-ce63-478a-8bd4-31f076bb2320 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:22.314992 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1392/2745] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:09:22 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:22.319278 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700741f-b512-4f0f-b5a7-4ccb89f4ba9a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:22.333679 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3700741f-b512-4f0f-b5a7-4ccb89f4ba9a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:22.341863 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54511e-692d-4b8d-9842-beb929c6aa7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:22.342537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54511e-692d-4b8d-9842-beb929c6aa7f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.342959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54511e-692d-4b8d-9842-beb929c6aa7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.343070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54511e-692d-4b8d-9842-beb929c6aa7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:22.343730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54511e-692d-4b8d-9842-beb929c6aa7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:22.346211 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54511e-692d-4b8d-9842-beb929c6aa7f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:22.359689 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1354/2746] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:22.364573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079bb6f-ce63-478a-8bd4-31f076bb2320 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:22.378638 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079bb6f-ce63-478a-8bd4-31f076bb2320 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:22.383943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d2127-c7ee-43fc-8b64-92f2b451e98a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:22.384850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d2127-c7ee-43fc-8b64-92f2b451e98a admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.385196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d2127-c7ee-43fc-8b64-92f2b451e98a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.385534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d2127-c7ee-43fc-8b64-92f2b451e98a admin admin] PATH_INFO: `/v3/projects/a84f3fd266b14cd382086c2a11aab2d2/users/b87f2eba32d3400e90180962337c18f7/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:22.403523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d2127-c7ee-43fc-8b64-92f2b451e98a admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84f3fd266b14cd382086c2a11aab2d2, user_id=b87f2eba32d3400e90180962337c18f7, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:22.406646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d2127-c7ee-43fc-8b64-92f2b451e98a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:22.421520 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1393/2747] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:22 2022] PUT /identity/v3/projects/a84f3fd266b14cd382086c2a11aab2d2/users/b87f2eba32d3400e90180962337c18f7/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:09:22.427228 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54511e-692d-4b8d-9842-beb929c6aa7f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:22.443048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54511e-692d-4b8d-9842-beb929c6aa7f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:22.504983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee19f5f6-6bf2-476a-9f86-a20e8b88c00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:22.505709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f5f6-6bf2-476a-9f86-a20e8b88c00f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.506157 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f5f6-6bf2-476a-9f86-a20e8b88c00f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.506508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee19f5f6-6bf2-476a-9f86-a20e8b88c00f admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:22.507240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19f5f6-6bf2-476a-9f86-a20e8b88c00f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:22.509055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee19f5f6-6bf2-476a-9f86-a20e8b88c00f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:22.520643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1355/2748] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:22 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:22.524517 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d2127-c7ee-43fc-8b64-92f2b451e98a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:22.539383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d2127-c7ee-43fc-8b64-92f2b451e98a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:22.545745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09055d0-80cb-43dc-a310-b60efe93a961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:22.546598 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09055d0-80cb-43dc-a310-b60efe93a961 admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.547077 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09055d0-80cb-43dc-a310-b60efe93a961 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.547936 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09055d0-80cb-43dc-a310-b60efe93a961 admin admin] PATH_INFO: `/v3/projects/a84f3fd266b14cd382086c2a11aab2d2/users/b87f2eba32d3400e90180962337c18f7/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:22.551597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09055d0-80cb-43dc-a310-b60efe93a961 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a84f3fd266b14cd382086c2a11aab2d2, user_id=b87f2eba32d3400e90180962337c18f7, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:22.554814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09055d0-80cb-43dc-a310-b60efe93a961 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:22.577433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1394/2749] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:22 2022] PUT /identity/v3/projects/a84f3fd266b14cd382086c2a11aab2d2/users/b87f2eba32d3400e90180962337c18f7/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:22.589579 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7747b-d3a0-4a7f-9a29-ad0eedb1b5a5 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.589873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7747b-d3a0-4a7f-9a29-ad0eedb1b5a5 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.590171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab7747b-d3a0-4a7f-9a29-ad0eedb1b5a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:22.611968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:22.627481 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-dab7747b-d3a0-4a7f-9a29-ad0eedb1b5a5 None None] MFA Rules not processed for user `b87f2eba32d3400e90180962337c18f7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:22.628738 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-dab7747b-d3a0-4a7f-9a29-ad0eedb1b5a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:22.721188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1356/2750] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:22 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:22.734224 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09055d0-80cb-43dc-a310-b60efe93a961 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:22.737065 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b09055d0-80cb-43dc-a310-b60efe93a961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:22.813915 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09055d0-80cb-43dc-a310-b60efe93a961 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:22.818106 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de7bee-73c4-46d9-bcad-f62c75c15346 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:22.819109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de7bee-73c4-46d9-bcad-f62c75c15346 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.819109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de7bee-73c4-46d9-bcad-f62c75c15346 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.819398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de7bee-73c4-46d9-bcad-f62c75c15346 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:22.831297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de7bee-73c4-46d9-bcad-f62c75c15346 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:22.832750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de7bee-73c4-46d9-bcad-f62c75c15346 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:22.845772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1395/2751] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:22 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:22.932750 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7747b-d3a0-4a7f-9a29-ad0eedb1b5a5 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:22.942626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab7747b-d3a0-4a7f-9a29-ad0eedb1b5a5 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:22.946257 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7af2cf-0405-404f-a01c-3a6371d7cc55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:22.946869 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7af2cf-0405-404f-a01c-3a6371d7cc55 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:22.947149 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7af2cf-0405-404f-a01c-3a6371d7cc55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:22.947420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7af2cf-0405-404f-a01c-3a6371d7cc55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:23.011241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7af2cf-0405-404f-a01c-3a6371d7cc55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:23.012699 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7af2cf-0405-404f-a01c-3a6371d7cc55 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:23.027615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1357/2752] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:24.360256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de7bee-73c4-46d9-bcad-f62c75c15346 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:24.364102 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c0de7bee-73c4-46d9-bcad-f62c75c15346 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:24.422599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:24.453872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de7bee-73c4-46d9-bcad-f62c75c15346 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:24.459429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cde14-5a7e-41de-8b94-cdd4a4a30ccb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:24.460095 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cde14-5a7e-41de-8b94-cdd4a4a30ccb service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:24.460373 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cde14-5a7e-41de-8b94-cdd4a4a30ccb service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:24.460647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6cde14-5a7e-41de-8b94-cdd4a4a30ccb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:24.462432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8b6cde14-5a7e-41de-8b94-cdd4a4a30ccb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:24.476672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cde14-5a7e-41de-8b94-cdd4a4a30ccb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:24.478974 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6cde14-5a7e-41de-8b94-cdd4a4a30ccb service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:24.489413 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1396/2753] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:24 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:09:27.727586 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7af2cf-0405-404f-a01c-3a6371d7cc55 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:27.728968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-1e7af2cf-0405-404f-a01c-3a6371d7cc55 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:27.821217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7af2cf-0405-404f-a01c-3a6371d7cc55 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:27.826040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:27.826765 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:27.827069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:27.827339 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] PATH_INFO: `/v3/users/a04d90d3f1e04a00b2ace1893ce5ef71` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:27.828539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a04d90d3f1e04a00b2ace1893ce5ef71)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:27.830597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:27.870873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:27.885509 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a04d90d3f1e04a00b2ace1893ce5ef71'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:27.895390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a04d90d3f1e04a00b2ace1893ce5ef71'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:27.908241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a04d90d3f1e04a00b2ace1893ce5ef71'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:27.908874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a04d90d3f1e04a00b2ace1893ce5ef71'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:27.916491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a04d90d3f1e04a00b2ace1893ce5ef71'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:27.922797 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1358/2754] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:27 2022] DELETE /identity/v3/users/a04d90d3f1e04a00b2ace1893ce5ef71 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:28.109272 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cde14-5a7e-41de-8b94-cdd4a4a30ccb service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:28.121345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6cde14-5a7e-41de-8b94-cdd4a4a30ccb service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:28.183370 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:28.184016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:28.184297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:28.184570 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] PATH_INFO: `/v3/projects/8e08bdf8d28f4a5093df2c5b26ccb9e6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:28.185758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8e08bdf8d28f4a5093df2c5b26ccb9e6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:28.187762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:28.235834 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8e08bdf8d28f4a5093df2c5b26ccb9e6', 'request_id': 'req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.241951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8e08bdf8d28f4a5093df2c5b26ccb9e6', 'request_id': 'req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.248137 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8e08bdf8d28f4a5093df2c5b26ccb9e6', 'request_id': 'req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.262383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] The token cache is being invalidate because project 8e08bdf8d28f4a5093df2c5b26ccb9e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:28.262675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.263316 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.264826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1397/2755] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:28 2022] DELETE /identity/v3/projects/8e08bdf8d28f4a5093df2c5b26ccb9e6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:28.269225 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:28.270722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:28.284301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e905c19b-0df9-43a3-9ce3-419fa3e6da3c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:28.347371 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:28.348365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:28.348783 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:28.349198 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] PATH_INFO: `/v3/users/4f1e83dce28e4ca6b1e5cd237347e572` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:28.350914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f1e83dce28e4ca6b1e5cd237347e572)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:28.352993 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:28.402697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f1e83dce28e4ca6b1e5cd237347e572'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.411041 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f1e83dce28e4ca6b1e5cd237347e572'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.423235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f1e83dce28e4ca6b1e5cd237347e572'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.423868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f1e83dce28e4ca6b1e5cd237347e572'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.431424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f1e83dce28e4ca6b1e5cd237347e572'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.437346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1359/2756] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:28 2022] DELETE /identity/v3/users/4f1e83dce28e4ca6b1e5cd237347e572 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:28.638179 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:28.639855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:28.655041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3ef83-8a5b-4bd5-ab5c-d5fc2241d7a1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:28.715748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:28.723481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:28.724199 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:28.724538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:28.724940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] PATH_INFO: `/v3/projects/4f52f8d9a593435da929e9ec5bfbfdd4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:28.726667 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f52f8d9a593435da929e9ec5bfbfdd4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:28.728095 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:28.782341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f52f8d9a593435da929e9ec5bfbfdd4', 'request_id': 'req-e0f67d0b-6b97-44db-857f-5690868df7ed'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.789156 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f52f8d9a593435da929e9ec5bfbfdd4', 'request_id': 'req-e0f67d0b-6b97-44db-857f-5690868df7ed'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.795202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f52f8d9a593435da929e9ec5bfbfdd4', 'request_id': 'req-e0f67d0b-6b97-44db-857f-5690868df7ed'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.809500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] The token cache is being invalidate because project 4f52f8d9a593435da929e9ec5bfbfdd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:28.809790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.810456 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e0f67d0b-6b97-44db-857f-5690868df7ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:28.811975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1398/2757] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:28 2022] DELETE /identity/v3/projects/4f52f8d9a593435da929e9ec5bfbfdd4 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:38.387887 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:38.389474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:38.402837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8a6690-5b25-446e-95d8-424881192cc8 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:38.429370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:38.464031 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:38.464712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:38.464995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:38.465267 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] PATH_INFO: `/v3/users/0478bb1f87b74eb8b9765068dfc8efc8` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:38.466575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0478bb1f87b74eb8b9765068dfc8efc8)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:38.468527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:38.522355 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0478bb1f87b74eb8b9765068dfc8efc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:38.531916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0478bb1f87b74eb8b9765068dfc8efc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:38.544774 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0478bb1f87b74eb8b9765068dfc8efc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:38.545778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0478bb1f87b74eb8b9765068dfc8efc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:38.552477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0478bb1f87b74eb8b9765068dfc8efc8'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:38.559838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1360/2758] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:38 2022] DELETE /identity/v3/users/0478bb1f87b74eb8b9765068dfc8efc8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:38.564473 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f25e8-b1fe-4c64-ab9b-db6c93499c95 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:38.565013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f25e8-b1fe-4c64-ab9b-db6c93499c95 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:38.565013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f25e8-b1fe-4c64-ab9b-db6c93499c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:38.585383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-883f25e8-b1fe-4c64-ab9b-db6c93499c95 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:38.586895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-883f25e8-b1fe-4c64-ab9b-db6c93499c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:38.647904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1399/2759] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:38.657914 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:38.659385 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:38.750103 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc636e7-edf8-4b73-8fef-172e8c1c7519 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:38.753817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc5d15-4e75-45d2-a845-e7fae2285033 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:38.754461 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc5d15-4e75-45d2-a845-e7fae2285033 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:38.754741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc5d15-4e75-45d2-a845-e7fae2285033 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:38.755011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc5d15-4e75-45d2-a845-e7fae2285033 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:38.764792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc5d15-4e75-45d2-a845-e7fae2285033 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:38.766269 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc5d15-4e75-45d2-a845-e7fae2285033 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:38.775920 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1361/2760] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:09:39.067475 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f25e8-b1fe-4c64-ab9b-db6c93499c95 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:39.068945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-883f25e8-b1fe-4c64-ab9b-db6c93499c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:39.081816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f25e8-b1fe-4c64-ab9b-db6c93499c95 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:39.087922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:39.088567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:39.088844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:39.089110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] PATH_INFO: `/v3/projects/0ccb24c2806740f1b2f463da0ae426f4` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:39.090361 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ccb24c2806740f1b2f463da0ae426f4)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:39.092293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:39.141697 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ccb24c2806740f1b2f463da0ae426f4', 'request_id': 'req-76bd161d-507e-4732-9d25-9c0b1d07e53f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.147726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ccb24c2806740f1b2f463da0ae426f4', 'request_id': 'req-76bd161d-507e-4732-9d25-9c0b1d07e53f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.153923 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ccb24c2806740f1b2f463da0ae426f4', 'request_id': 'req-76bd161d-507e-4732-9d25-9c0b1d07e53f'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.168743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] The token cache is being invalidate because project 0ccb24c2806740f1b2f463da0ae426f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:39.168967 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.169602 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.171141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1400/2761] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:39 2022] DELETE /identity/v3/projects/0ccb24c2806740f1b2f463da0ae426f4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:39.175263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc5d15-4e75-45d2-a845-e7fae2285033 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:39.186114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc5d15-4e75-45d2-a845-e7fae2285033 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:39.216395 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:39.246980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:39.247615 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:39.247888 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:39.248155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] PATH_INFO: `/v3/users/b87f2eba32d3400e90180962337c18f7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:39.249313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87f2eba32d3400e90180962337c18f7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:39.251282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:39.301735 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87f2eba32d3400e90180962337c18f7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.313935 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87f2eba32d3400e90180962337c18f7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.329777 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87f2eba32d3400e90180962337c18f7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.330486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87f2eba32d3400e90180962337c18f7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.337912 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6c8d1775-2c43-4a04-8e38-c2b263b8dd33 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87f2eba32d3400e90180962337c18f7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.344307 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1362/2762] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:39 2022] DELETE /identity/v3/users/b87f2eba32d3400e90180962337c18f7 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:39.635401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:39.637075 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:39.650918 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bd161d-507e-4732-9d25-9c0b1d07e53f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:39.659728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:39.713017 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:39.713574 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:39.713876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:39.714177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] PATH_INFO: `/v3/projects/a84f3fd266b14cd382086c2a11aab2d2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:39.715294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a84f3fd266b14cd382086c2a11aab2d2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:39.717225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:39.765368 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a84f3fd266b14cd382086c2a11aab2d2', 'request_id': 'req-e4d1f0f6-f472-4500-8f94-2e8c75851297'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.771566 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a84f3fd266b14cd382086c2a11aab2d2', 'request_id': 'req-e4d1f0f6-f472-4500-8f94-2e8c75851297'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.777383 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a84f3fd266b14cd382086c2a11aab2d2', 'request_id': 'req-e4d1f0f6-f472-4500-8f94-2e8c75851297'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.791945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] The token cache is being invalidate because project a84f3fd266b14cd382086c2a11aab2d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:39.792236 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.792849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-e4d1f0f6-f472-4500-8f94-2e8c75851297 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:39.794359 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1401/2763] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:39 2022] DELETE /identity/v3/projects/a84f3fd266b14cd382086c2a11aab2d2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:39.800477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fe27b-5189-46e0-a312-a33bf9a39f00 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:39.801005 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fe27b-5189-46e0-a312-a33bf9a39f00 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:39.801124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545fe27b-5189-46e0-a312-a33bf9a39f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:39.822728 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-545fe27b-5189-46e0-a312-a33bf9a39f00 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:39.823934 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-545fe27b-5189-46e0-a312-a33bf9a39f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:39.894690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1363/2764] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:09:39.905115 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eb8dd-7fef-40f6-8a02-32f6689c81d8 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:39.905474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eb8dd-7fef-40f6-8a02-32f6689c81d8 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:39.905815 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eb8dd-7fef-40f6-8a02-32f6689c81d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:39.934908 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b91eb8dd-7fef-40f6-8a02-32f6689c81d8 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:39.936127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b91eb8dd-7fef-40f6-8a02-32f6689c81d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:39.946064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1402/2765] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:39.952585 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fe27b-5189-46e0-a312-a33bf9a39f00 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:39.963626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545fe27b-5189-46e0-a312-a33bf9a39f00 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:39.969488 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfcf17-9fbb-4300-89b9-acff2ed9d18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:39.970109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfcf17-9fbb-4300-89b9-acff2ed9d18b admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:39.970450 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfcf17-9fbb-4300-89b9-acff2ed9d18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:39.970729 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfcf17-9fbb-4300-89b9-acff2ed9d18b admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:39.971383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfcf17-9fbb-4300-89b9-acff2ed9d18b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:09:39.971690 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfcf17-9fbb-4300-89b9-acff2ed9d18b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:39.973291 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfcf17-9fbb-4300-89b9-acff2ed9d18b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:39.986244 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1364/2766] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:09:39.990647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eb8dd-7fef-40f6-8a02-32f6689c81d8 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.000674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eb8dd-7fef-40f6-8a02-32f6689c81d8 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.006752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c6d96-a7b1-4763-b8e7-f80e2fafa8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.007356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c6d96-a7b1-4763-b8e7-f80e2fafa8a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.007634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c6d96-a7b1-4763-b8e7-f80e2fafa8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.008020 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c6d96-a7b1-4763-b8e7-f80e2fafa8a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.008709 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c6d96-a7b1-4763-b8e7-f80e2fafa8a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.010672 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c6d96-a7b1-4763-b8e7-f80e2fafa8a7 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.042674 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1403/2767] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:09:39 2022] POST /identity/v3/projects => generated 401 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:40.049114 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfcf17-9fbb-4300-89b9-acff2ed9d18b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.065050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfcf17-9fbb-4300-89b9-acff2ed9d18b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.103184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:40.129645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379326c-2e17-403d-8676-fe187f3c1b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.130300 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379326c-2e17-403d-8676-fe187f3c1b85 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.130611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379326c-2e17-403d-8676-fe187f3c1b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.130882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2379326c-2e17-403d-8676-fe187f3c1b85 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.131547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379326c-2e17-403d-8676-fe187f3c1b85 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.133665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2379326c-2e17-403d-8676-fe187f3c1b85 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.161671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1365/2768] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:09:40 2022] POST /identity/v3/users => generated 301 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:40.166293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c6d96-a7b1-4763-b8e7-f80e2fafa8a7 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.177056 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c6d96-a7b1-4763-b8e7-f80e2fafa8a7 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.182489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f7817-43ad-4113-ad6a-7a9dfdb2a6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.183094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f7817-43ad-4113-ad6a-7a9dfdb2a6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.183377 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f7817-43ad-4113-ad6a-7a9dfdb2a6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.183653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc2f7817-43ad-4113-ad6a-7a9dfdb2a6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.184293 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f7817-43ad-4113-ad6a-7a9dfdb2a6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.185910 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc2f7817-43ad-4113-ad6a-7a9dfdb2a6b1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.196244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1404/2769] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:40.200270 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379326c-2e17-403d-8676-fe187f3c1b85 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.210261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2379326c-2e17-403d-8676-fe187f3c1b85 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.215346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdd3f1b-c006-4a07-8baf-a562c668b227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.215882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd3f1b-c006-4a07-8baf-a562c668b227 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.216160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd3f1b-c006-4a07-8baf-a562c668b227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.216430 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdd3f1b-c006-4a07-8baf-a562c668b227 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.217066 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd3f1b-c006-4a07-8baf-a562c668b227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.218762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdd3f1b-c006-4a07-8baf-a562c668b227 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.228950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1366/2770] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:40.233505 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f7817-43ad-4113-ad6a-7a9dfdb2a6b1 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.248525 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc2f7817-43ad-4113-ad6a-7a9dfdb2a6b1 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.257458 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45665ef8-ae0f-4c34-b9b1-a2329c8f3cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.258546 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45665ef8-ae0f-4c34-b9b1-a2329c8f3cbd admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.258845 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45665ef8-ae0f-4c34-b9b1-a2329c8f3cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.259125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45665ef8-ae0f-4c34-b9b1-a2329c8f3cbd admin admin] PATH_INFO: `/v3/projects/eb385c8c5c4246b396ab1660d80cbe3c/users/41d1364b5c484b1a8b83e359e9befdc7/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.275273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45665ef8-ae0f-4c34-b9b1-a2329c8f3cbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb385c8c5c4246b396ab1660d80cbe3c, user_id=41d1364b5c484b1a8b83e359e9befdc7, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.277940 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45665ef8-ae0f-4c34-b9b1-a2329c8f3cbd admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.293969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1405/2771] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:40 2022] PUT /identity/v3/projects/eb385c8c5c4246b396ab1660d80cbe3c/users/41d1364b5c484b1a8b83e359e9befdc7/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:09:40.304516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaba4e8-56c7-4dd5-8ada-7fee8381c730 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.304851 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaba4e8-56c7-4dd5-8ada-7fee8381c730 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.305800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaba4e8-56c7-4dd5-8ada-7fee8381c730 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.340663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bcaba4e8-56c7-4dd5-8ada-7fee8381c730 None None] MFA Rules not processed for user `41d1364b5c484b1a8b83e359e9befdc7`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:40.342497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bcaba4e8-56c7-4dd5-8ada-7fee8381c730 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:40.415552 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1367/2772] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:40 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:40.420828 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45665ef8-ae0f-4c34-b9b1-a2329c8f3cbd admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.478146 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:40.502313 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45665ef8-ae0f-4c34-b9b1-a2329c8f3cbd admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.507865 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe08f7d-1edb-42ad-bf52-c92997702860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.508492 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08f7d-1edb-42ad-bf52-c92997702860 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.508796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08f7d-1edb-42ad-bf52-c92997702860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.509063 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe08f7d-1edb-42ad-bf52-c92997702860 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.509693 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe08f7d-1edb-42ad-bf52-c92997702860 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.511647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe08f7d-1edb-42ad-bf52-c92997702860 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.540562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1406/2773] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:09:40 2022] POST /identity/v3/projects => generated 401 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:40.544852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaba4e8-56c7-4dd5-8ada-7fee8381c730 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.556899 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaba4e8-56c7-4dd5-8ada-7fee8381c730 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.622516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d882e-6b58-41ad-8a9c-8eaa80e991ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.623138 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d882e-6b58-41ad-8a9c-8eaa80e991ea admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.623423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d882e-6b58-41ad-8a9c-8eaa80e991ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.623691 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914d882e-6b58-41ad-8a9c-8eaa80e991ea admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.624327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d882e-6b58-41ad-8a9c-8eaa80e991ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.626290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914d882e-6b58-41ad-8a9c-8eaa80e991ea admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.650645 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1368/2774] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:09:40 2022] POST /identity/v3/users => generated 307 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:40.654817 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe08f7d-1edb-42ad-bf52-c92997702860 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.665655 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe08f7d-1edb-42ad-bf52-c92997702860 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.671335 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10d380-684b-4344-894a-3689eeda2f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.671969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10d380-684b-4344-894a-3689eeda2f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.672254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10d380-684b-4344-894a-3689eeda2f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.672526 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d10d380-684b-4344-894a-3689eeda2f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.673157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10d380-684b-4344-894a-3689eeda2f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.674826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d10d380-684b-4344-894a-3689eeda2f04 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.686586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1407/2775] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:40.691286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d882e-6b58-41ad-8a9c-8eaa80e991ea admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.702200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914d882e-6b58-41ad-8a9c-8eaa80e991ea admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.707390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecf56a-6d6b-48c5-9dce-0175dbeeaf6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.708049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecf56a-6d6b-48c5-9dce-0175dbeeaf6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.708325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecf56a-6d6b-48c5-9dce-0175dbeeaf6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.708602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecf56a-6d6b-48c5-9dce-0175dbeeaf6f admin admin] PATH_INFO: `/v3/projects/5ef83fc2eb6448a6a3e3dac04ac22c11/users/6220e8dd37604c148cc0c4cd14f06cbc/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.725347 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecf56a-6d6b-48c5-9dce-0175dbeeaf6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef83fc2eb6448a6a3e3dac04ac22c11, user_id=6220e8dd37604c148cc0c4cd14f06cbc, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.728217 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eecf56a-6d6b-48c5-9dce-0175dbeeaf6f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.746639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1369/2776] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:40 2022] PUT /identity/v3/projects/5ef83fc2eb6448a6a3e3dac04ac22c11/users/6220e8dd37604c148cc0c4cd14f06cbc/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:09:40.750809 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10d380-684b-4344-894a-3689eeda2f04 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.761052 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d10d380-684b-4344-894a-3689eeda2f04 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.819494 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4004384d-dba3-4c5a-a00f-722c015da228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.820100 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004384d-dba3-4c5a-a00f-722c015da228 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.820442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004384d-dba3-4c5a-a00f-722c015da228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.820754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4004384d-dba3-4c5a-a00f-722c015da228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.821398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004384d-dba3-4c5a-a00f-722c015da228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.823044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4004384d-dba3-4c5a-a00f-722c015da228 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.832769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1408/2777] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:40.837627 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecf56a-6d6b-48c5-9dce-0175dbeeaf6f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:40.848491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecf56a-6d6b-48c5-9dce-0175dbeeaf6f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:40.855537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd2c6b-dfff-4f41-84e7-808472a72202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:40.856541 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd2c6b-dfff-4f41-84e7-808472a72202 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.856959 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd2c6b-dfff-4f41-84e7-808472a72202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.857327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bd2c6b-dfff-4f41-84e7-808472a72202 admin admin] PATH_INFO: `/v3/projects/5ef83fc2eb6448a6a3e3dac04ac22c11/users/6220e8dd37604c148cc0c4cd14f06cbc/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.860703 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd2c6b-dfff-4f41-84e7-808472a72202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef83fc2eb6448a6a3e3dac04ac22c11, user_id=6220e8dd37604c148cc0c4cd14f06cbc, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:40.864983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bd2c6b-dfff-4f41-84e7-808472a72202 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:40.873143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:40.881908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1370/2778] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:40 2022] PUT /identity/v3/projects/5ef83fc2eb6448a6a3e3dac04ac22c11/users/6220e8dd37604c148cc0c4cd14f06cbc/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:40.893998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e485f4-e778-45ee-abd0-498332a967b7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:40.894320 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e485f4-e778-45ee-abd0-498332a967b7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:40.894597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e485f4-e778-45ee-abd0-498332a967b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:40.924965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-b1e485f4-e778-45ee-abd0-498332a967b7 None None] MFA Rules not processed for user `6220e8dd37604c148cc0c4cd14f06cbc`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:40.926218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b1e485f4-e778-45ee-abd0-498332a967b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:41.002691 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1409/2779] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:40 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:41.016213 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd2c6b-dfff-4f41-84e7-808472a72202 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:41.019262 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-65bd2c6b-dfff-4f41-84e7-808472a72202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:41.109702 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bd2c6b-dfff-4f41-84e7-808472a72202 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:41.114040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec408f-bf08-4b77-9b9f-76b000855832 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:41.114685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec408f-bf08-4b77-9b9f-76b000855832 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:41.114968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec408f-bf08-4b77-9b9f-76b000855832 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:41.115237 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dec408f-bf08-4b77-9b9f-76b000855832 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:41.125402 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec408f-bf08-4b77-9b9f-76b000855832 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:41.126889 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dec408f-bf08-4b77-9b9f-76b000855832 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:41.137207 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1371/2780] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:41 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:41.218730 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e485f4-e778-45ee-abd0-498332a967b7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:41.229479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e485f4-e778-45ee-abd0-498332a967b7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:41.233428 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7352b-e5ac-40b6-9071-f5260a621dbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:41.234034 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7352b-e5ac-40b6-9071-f5260a621dbe service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:41.234343 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7352b-e5ac-40b6-9071-f5260a621dbe service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:41.234614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b7352b-e5ac-40b6-9071-f5260a621dbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:41.265385 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:41.293646 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7352b-e5ac-40b6-9071-f5260a621dbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:41.295302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b7352b-e5ac-40b6-9071-f5260a621dbe service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:41.306855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1410/2781] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:41 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:45.691155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec408f-bf08-4b77-9b9f-76b000855832 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:45.759946 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dec408f-bf08-4b77-9b9f-76b000855832 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:45.765082 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:45.765764 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:45.766037 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:45.766346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] PATH_INFO: `/v3/users/41d1364b5c484b1a8b83e359e9befdc7` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:45.767524 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41d1364b5c484b1a8b83e359e9befdc7)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:45.769470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:45.795290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7352b-e5ac-40b6-9071-f5260a621dbe service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:45.805257 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b7352b-e5ac-40b6-9071-f5260a621dbe service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:45.810506 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:45.811159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:45.811457 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:45.811728 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:45.813514 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:45.823320 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41d1364b5c484b1a8b83e359e9befdc7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:45.832512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41d1364b5c484b1a8b83e359e9befdc7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:45.844626 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41d1364b5c484b1a8b83e359e9befdc7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:45.845292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41d1364b5c484b1a8b83e359e9befdc7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:45.851966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0ba61c20-59f5-4ba2-ae17-cb53a964c448 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41d1364b5c484b1a8b83e359e9befdc7'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:45.853992 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:45.858710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1372/2782] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:45 2022] DELETE /identity/v3/users/41d1364b5c484b1a8b83e359e9befdc7 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:45.863226 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f3d7c-0749-46f6-9c68-2f18e4a0bcb1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:45.863554 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f3d7c-0749-46f6-9c68-2f18e4a0bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:45.863817 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7f3d7c-0749-46f6-9c68-2f18e4a0bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:45.877328 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:45.878933 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:45.884100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-fb7f3d7c-0749-46f6-9c68-2f18e4a0bcb1 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:45.885323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb7f3d7c-0749-46f6-9c68-2f18e4a0bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:45.889282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1411/2783] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:45.947808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1373/2784] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:45.958289 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:45.959866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:46.022486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9f5a32-b05a-4afa-8a67-0a3024201b4c service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:46.087013 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f5a6b-00df-43d6-a7c1-020733243f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:46.087641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f5a6b-00df-43d6-a7c1-020733243f6c service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:46.087922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f5a6b-00df-43d6-a7c1-020733243f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:46.088194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4f5a6b-00df-43d6-a7c1-020733243f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:46.099016 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f5a6b-00df-43d6-a7c1-020733243f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:46.100768 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4f5a6b-00df-43d6-a7c1-020733243f6c service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:46.111698 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1412/2785] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:46.388753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f3d7c-0749-46f6-9c68-2f18e4a0bcb1 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:46.390252 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-fb7f3d7c-0749-46f6-9c68-2f18e4a0bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:46.404049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7f3d7c-0749-46f6-9c68-2f18e4a0bcb1 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:46.409203 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:46.409855 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:46.410168 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:46.410447 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] PATH_INFO: `/v3/projects/eb385c8c5c4246b396ab1660d80cbe3c` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:46.411580 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb385c8c5c4246b396ab1660d80cbe3c)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:46.413546 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:46.464779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb385c8c5c4246b396ab1660d80cbe3c', 'request_id': 'req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.471378 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb385c8c5c4246b396ab1660d80cbe3c', 'request_id': 'req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.477542 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb385c8c5c4246b396ab1660d80cbe3c', 'request_id': 'req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.492115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] The token cache is being invalidate because project eb385c8c5c4246b396ab1660d80cbe3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:46.492400 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.493029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.494565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1374/2786] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:46 2022] DELETE /identity/v3/projects/eb385c8c5c4246b396ab1660d80cbe3c => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:46.499572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f5a6b-00df-43d6-a7c1-020733243f6c service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:46.509308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4f5a6b-00df-43d6-a7c1-020733243f6c service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:46.540164 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:46.575329 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:46.575880 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:46.576160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:46.576435 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] PATH_INFO: `/v3/users/6220e8dd37604c148cc0c4cd14f06cbc` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:46.577584 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=6220e8dd37604c148cc0c4cd14f06cbc)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:46.579543 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:46.636994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6220e8dd37604c148cc0c4cd14f06cbc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.637852 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6220e8dd37604c148cc0c4cd14f06cbc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.646793 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6220e8dd37604c148cc0c4cd14f06cbc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.659105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6220e8dd37604c148cc0c4cd14f06cbc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.659772 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-0c522968-e8d2-4f27-9d03-2213f2cc26ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6220e8dd37604c148cc0c4cd14f06cbc'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:46.674309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1413/2787] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:46 2022] DELETE /identity/v3/users/6220e8dd37604c148cc0c4cd14f06cbc => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:46.959457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:46.961715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:46.979216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eca21e9-e1f8-4fa6-9c26-ea398bc69c50 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.040256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.040898 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.041182 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.041460 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] PATH_INFO: `/v3/projects/5ef83fc2eb6448a6a3e3dac04ac22c11` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.042648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ef83fc2eb6448a6a3e3dac04ac22c11)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.044600 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:47.077963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:47.096127 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ef83fc2eb6448a6a3e3dac04ac22c11', 'request_id': 'req-f26b3a9b-7804-42c8-8020-c220aae74d80'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:47.102941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ef83fc2eb6448a6a3e3dac04ac22c11', 'request_id': 'req-f26b3a9b-7804-42c8-8020-c220aae74d80'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:47.109939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ef83fc2eb6448a6a3e3dac04ac22c11', 'request_id': 'req-f26b3a9b-7804-42c8-8020-c220aae74d80'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:47.128548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] The token cache is being invalidate because project 5ef83fc2eb6448a6a3e3dac04ac22c11 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:47.128840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:47.129480 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-f26b3a9b-7804-42c8-8020-c220aae74d80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:47.131308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1375/2788] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:46 2022] DELETE /identity/v3/projects/5ef83fc2eb6448a6a3e3dac04ac22c11 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:47.136043 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888db85-1f00-4c84-9990-5380043cd3ff None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.136565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888db85-1f00-4c84-9990-5380043cd3ff None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.136690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3888db85-1f00-4c84-9990-5380043cd3ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.157760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3888db85-1f00-4c84-9990-5380043cd3ff None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:47.159006 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3888db85-1f00-4c84-9990-5380043cd3ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:47.222015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1414/2789] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:09:47.230094 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215263c-f71e-40c4-8098-073f3dd85fc4 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.230383 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215263c-f71e-40c4-8098-073f3dd85fc4 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.230655 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f215263c-f71e-40c4-8098-073f3dd85fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.251642 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f215263c-f71e-40c4-8098-073f3dd85fc4 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:47.252911 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f215263c-f71e-40c4-8098-073f3dd85fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:47.261700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1376/2790] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:47.266344 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888db85-1f00-4c84-9990-5380043cd3ff None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:47.276634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3888db85-1f00-4c84-9990-5380043cd3ff None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.282044 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56486070-dd6d-4c89-adbb-8fdcf9cc7bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.282690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56486070-dd6d-4c89-adbb-8fdcf9cc7bcf admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.282969 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56486070-dd6d-4c89-adbb-8fdcf9cc7bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.283241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56486070-dd6d-4c89-adbb-8fdcf9cc7bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.283895 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56486070-dd6d-4c89-adbb-8fdcf9cc7bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:09:47.284202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56486070-dd6d-4c89-adbb-8fdcf9cc7bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.285829 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56486070-dd6d-4c89-adbb-8fdcf9cc7bcf admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:47.297960 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1415/2791] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:09:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:09:47.302327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215263c-f71e-40c4-8098-073f3dd85fc4 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:47.312219 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f215263c-f71e-40c4-8098-073f3dd85fc4 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.317664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c4353-4f88-45cd-90ac-bc78bddc74cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.318311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c4353-4f88-45cd-90ac-bc78bddc74cc admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.318594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c4353-4f88-45cd-90ac-bc78bddc74cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.318872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85c4353-4f88-45cd-90ac-bc78bddc74cc admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.319518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c4353-4f88-45cd-90ac-bc78bddc74cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.321455 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85c4353-4f88-45cd-90ac-bc78bddc74cc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:47.355113 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1377/2792] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:09:47 2022] POST /identity/v3/projects => generated 389 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:47.360074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56486070-dd6d-4c89-adbb-8fdcf9cc7bcf admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:47.370273 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56486070-dd6d-4c89-adbb-8fdcf9cc7bcf admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.406041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:47.449380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f81aeb-9051-44f8-8c2d-3c05b7e4a98f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.450058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f81aeb-9051-44f8-8c2d-3c05b7e4a98f admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.450367 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f81aeb-9051-44f8-8c2d-3c05b7e4a98f admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.450631 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f81aeb-9051-44f8-8c2d-3c05b7e4a98f admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.451264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f81aeb-9051-44f8-8c2d-3c05b7e4a98f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.453186 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f81aeb-9051-44f8-8c2d-3c05b7e4a98f admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:47.476988 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1416/2793] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:09:47 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:47.481708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c4353-4f88-45cd-90ac-bc78bddc74cc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:47.492421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85c4353-4f88-45cd-90ac-bc78bddc74cc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.497274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72b740-144d-4018-93e1-d95012778cfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.497897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72b740-144d-4018-93e1-d95012778cfc admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.498206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72b740-144d-4018-93e1-d95012778cfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.498483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c72b740-144d-4018-93e1-d95012778cfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.499132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72b740-144d-4018-93e1-d95012778cfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.500747 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c72b740-144d-4018-93e1-d95012778cfc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:47.510996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1378/2794] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:09:47.515432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f81aeb-9051-44f8-8c2d-3c05b7e4a98f admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:47.525626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f81aeb-9051-44f8-8c2d-3c05b7e4a98f admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.530632 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd06957-fd58-48a1-9f7e-4d167959a135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.531265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd06957-fd58-48a1-9f7e-4d167959a135 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.531545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd06957-fd58-48a1-9f7e-4d167959a135 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.531816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd06957-fd58-48a1-9f7e-4d167959a135 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.532433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd06957-fd58-48a1-9f7e-4d167959a135 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.534039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd06957-fd58-48a1-9f7e-4d167959a135 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:47.544694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1417/2795] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:47.549255 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72b740-144d-4018-93e1-d95012778cfc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:47.560166 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c72b740-144d-4018-93e1-d95012778cfc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.566049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef673-d1f4-422a-b885-b6a66d943547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.566730 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef673-d1f4-422a-b885-b6a66d943547 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.567006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef673-d1f4-422a-b885-b6a66d943547 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.567274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ef673-d1f4-422a-b885-b6a66d943547 admin admin] PATH_INFO: `/v3/projects/531ee321e0564b4d95a138fdc007770a/users/e1a039d2659b4457818c6cbb0720e87f/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.583574 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ef673-d1f4-422a-b885-b6a66d943547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=531ee321e0564b4d95a138fdc007770a, user_id=e1a039d2659b4457818c6cbb0720e87f, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.586458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442ef673-d1f4-422a-b885-b6a66d943547 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:47.601929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1379/2796] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:47 2022] PUT /identity/v3/projects/531ee321e0564b4d95a138fdc007770a/users/e1a039d2659b4457818c6cbb0720e87f/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:47.613991 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a65f-d447-4f99-a1d3-d8d178b68cd9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.614318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a65f-d447-4f99-a1d3-d8d178b68cd9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.614601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d7a65f-d447-4f99-a1d3-d8d178b68cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.643899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-61d7a65f-d447-4f99-a1d3-d8d178b68cd9 None None] MFA Rules not processed for user `e1a039d2659b4457818c6cbb0720e87f`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:47.645123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-61d7a65f-d447-4f99-a1d3-d8d178b68cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:47.731024 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1418/2797] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:47 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:47.735925 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef673-d1f4-422a-b885-b6a66d943547 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:47.807039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ef673-d1f4-422a-b885-b6a66d943547 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.813155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050df799-0ba1-4ffc-8e67-90b593a821d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.813770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050df799-0ba1-4ffc-8e67-90b593a821d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.814048 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050df799-0ba1-4ffc-8e67-90b593a821d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.814345 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050df799-0ba1-4ffc-8e67-90b593a821d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.814952 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050df799-0ba1-4ffc-8e67-90b593a821d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.816862 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050df799-0ba1-4ffc-8e67-90b593a821d0 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:47.852151 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1380/2798] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:09:47 2022] POST /identity/v3/projects => generated 387 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:47.857874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7a65f-d447-4f99-a1d3-d8d178b68cd9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:47.870733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d7a65f-d447-4f99-a1d3-d8d178b68cd9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.928629 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df7e3-92aa-429e-a36b-97e4a31a4037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.929388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df7e3-92aa-429e-a36b-97e4a31a4037 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.929872 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df7e3-92aa-429e-a36b-97e4a31a4037 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.930356 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319df7e3-92aa-429e-a36b-97e4a31a4037 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.931782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319df7e3-92aa-429e-a36b-97e4a31a4037 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.935535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319df7e3-92aa-429e-a36b-97e4a31a4037 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:47.966379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1419/2799] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:09:47 2022] POST /identity/v3/users => generated 300 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:47.970584 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050df799-0ba1-4ffc-8e67-90b593a821d0 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:47.982421 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050df799-0ba1-4ffc-8e67-90b593a821d0 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:47.988399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cbe83-9ad4-4576-b0da-16854d0ca9f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:47.989009 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cbe83-9ad4-4576-b0da-16854d0ca9f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:47.989289 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cbe83-9ad4-4576-b0da-16854d0ca9f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:47.989559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855cbe83-9ad4-4576-b0da-16854d0ca9f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:47.990227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cbe83-9ad4-4576-b0da-16854d0ca9f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:47.991847 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855cbe83-9ad4-4576-b0da-16854d0ca9f9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:48.004110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1381/2800] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 13 07:09:48.009427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df7e3-92aa-429e-a36b-97e4a31a4037 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:48.029511 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319df7e3-92aa-429e-a36b-97e4a31a4037 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:48.035547 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d97873-233c-49b3-90d0-6c571296996d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:48.036191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d97873-233c-49b3-90d0-6c571296996d admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:48.036461 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d97873-233c-49b3-90d0-6c571296996d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:48.036723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d97873-233c-49b3-90d0-6c571296996d admin admin] PATH_INFO: `/v3/projects/53623f8a24e34065855fa6b6e03d6b2a/users/259f668fda3d4c39a59843c78e3eeafc/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:48.060827 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d97873-233c-49b3-90d0-6c571296996d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53623f8a24e34065855fa6b6e03d6b2a, user_id=259f668fda3d4c39a59843c78e3eeafc, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:48.063538 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d97873-233c-49b3-90d0-6c571296996d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:48.079914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1420/2801] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:48 2022] PUT /identity/v3/projects/53623f8a24e34065855fa6b6e03d6b2a/users/259f668fda3d4c39a59843c78e3eeafc/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:48.083985 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cbe83-9ad4-4576-b0da-16854d0ca9f9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:48.094819 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855cbe83-9ad4-4576-b0da-16854d0ca9f9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:48.117243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:48.157779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90855be-03dd-4f21-b1cb-90093dab50b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:48.158431 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90855be-03dd-4f21-b1cb-90093dab50b7 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:48.158710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90855be-03dd-4f21-b1cb-90093dab50b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:48.158984 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90855be-03dd-4f21-b1cb-90093dab50b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:48.159611 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90855be-03dd-4f21-b1cb-90093dab50b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:48.161212 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90855be-03dd-4f21-b1cb-90093dab50b7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:48.171184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1382/2802] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:48 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:48.177122 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d97873-233c-49b3-90d0-6c571296996d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:48.194760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d97873-233c-49b3-90d0-6c571296996d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:48.202157 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ab7f8-889e-4038-8cfa-237671c2bb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:48.202814 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ab7f8-889e-4038-8cfa-237671c2bb3b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:48.203094 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ab7f8-889e-4038-8cfa-237671c2bb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:48.203363 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821ab7f8-889e-4038-8cfa-237671c2bb3b admin admin] PATH_INFO: `/v3/projects/53623f8a24e34065855fa6b6e03d6b2a/users/259f668fda3d4c39a59843c78e3eeafc/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:48.205401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ab7f8-889e-4038-8cfa-237671c2bb3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=53623f8a24e34065855fa6b6e03d6b2a, user_id=259f668fda3d4c39a59843c78e3eeafc, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:48.208058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821ab7f8-889e-4038-8cfa-237671c2bb3b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:48.211314 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:48.221284 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1421/2803] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:48 2022] PUT /identity/v3/projects/53623f8a24e34065855fa6b6e03d6b2a/users/259f668fda3d4c39a59843c78e3eeafc/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:48.288820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf966736-2048-46af-a4a8-8e325395a92f None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:48.289104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf966736-2048-46af-a4a8-8e325395a92f None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:48.289367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf966736-2048-46af-a4a8-8e325395a92f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:48.321432 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-bf966736-2048-46af-a4a8-8e325395a92f None None] MFA Rules not processed for user `259f668fda3d4c39a59843c78e3eeafc`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:48.322677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-bf966736-2048-46af-a4a8-8e325395a92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:48.392870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1383/2804] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:48 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:48.410524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ab7f8-889e-4038-8cfa-237671c2bb3b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:48.412838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-821ab7f8-889e-4038-8cfa-237671c2bb3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:48.477251 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821ab7f8-889e-4038-8cfa-237671c2bb3b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:48.481572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701e8dee-1b30-4802-8c97-86cc852a70ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:48.482214 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e8dee-1b30-4802-8c97-86cc852a70ed service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:48.482514 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e8dee-1b30-4802-8c97-86cc852a70ed service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:48.482760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701e8dee-1b30-4802-8c97-86cc852a70ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:48.492564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e8dee-1b30-4802-8c97-86cc852a70ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:48.493987 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701e8dee-1b30-4802-8c97-86cc852a70ed service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:48.503520 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1422/2805] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:48 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:48.643648 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf966736-2048-46af-a4a8-8e325395a92f None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:48.713621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf966736-2048-46af-a4a8-8e325395a92f None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:48.718771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:48.719414 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:48.719740 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:48.720007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] PATH_INFO: `/v3/users/e1a039d2659b4457818c6cbb0720e87f` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:48.721137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1a039d2659b4457818c6cbb0720e87f)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:48.723115 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:48.780794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1a039d2659b4457818c6cbb0720e87f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:48.782503 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:48.791065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1a039d2659b4457818c6cbb0720e87f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:48.804486 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1a039d2659b4457818c6cbb0720e87f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:48.805153 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1a039d2659b4457818c6cbb0720e87f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:48.813099 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1a039d2659b4457818c6cbb0720e87f'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:48.819080 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1384/2806] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:48 2022] DELETE /identity/v3/users/e1a039d2659b4457818c6cbb0720e87f => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:48.823744 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662f170-382e-4366-9b4a-0a724b95c678 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:48.824028 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662f170-382e-4366-9b4a-0a724b95c678 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:48.824297 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0662f170-382e-4366-9b4a-0a724b95c678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:48.844064 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0662f170-382e-4366-9b4a-0a724b95c678 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:48.845270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0662f170-382e-4366-9b4a-0a724b95c678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:48.908364 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1423/2807] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:48.919173 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:48.920668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:48.981724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83daa8d2-91d6-4818-903c-44238c1123e5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:48.985939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89368e-c298-4455-bfe8-d2a82cd75323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:48.986711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89368e-c298-4455-bfe8-d2a82cd75323 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:48.986998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89368e-c298-4455-bfe8-d2a82cd75323 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:48.987284 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f89368e-c298-4455-bfe8-d2a82cd75323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:48.997058 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89368e-c298-4455-bfe8-d2a82cd75323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:48.998638 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f89368e-c298-4455-bfe8-d2a82cd75323 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:49.008899 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1385/2808] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:49.290804 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662f170-382e-4366-9b4a-0a724b95c678 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:49.292375 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0662f170-382e-4366-9b4a-0a724b95c678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:49.306625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0662f170-382e-4366-9b4a-0a724b95c678 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:49.312792 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:49.313478 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:49.313734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:49.314005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] PATH_INFO: `/v3/projects/531ee321e0564b4d95a138fdc007770a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:49.315294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] RBAC: Authorizing `identity:delete_project(project_id=531ee321e0564b4d95a138fdc007770a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:49.317649 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:49.369038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '531ee321e0564b4d95a138fdc007770a', 'request_id': 'req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.375760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '531ee321e0564b4d95a138fdc007770a', 'request_id': 'req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.381874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '531ee321e0564b4d95a138fdc007770a', 'request_id': 'req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.395743 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] The token cache is being invalidate because project 531ee321e0564b4d95a138fdc007770a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:49.396083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.396708 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.398504 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1424/2809] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:49 2022] DELETE /identity/v3/projects/531ee321e0564b4d95a138fdc007770a => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:49.402666 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89368e-c298-4455-bfe8-d2a82cd75323 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:49.426778 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f89368e-c298-4455-bfe8-d2a82cd75323 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:49.489327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:49.489966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:49.490273 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:49.490543 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] PATH_INFO: `/v3/users/259f668fda3d4c39a59843c78e3eeafc` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:49.491658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] RBAC: Authorizing `identity:delete_user(user_id=259f668fda3d4c39a59843c78e3eeafc)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:49.493566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:49.549235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '259f668fda3d4c39a59843c78e3eeafc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.560571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '259f668fda3d4c39a59843c78e3eeafc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.586724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:49.588188 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '259f668fda3d4c39a59843c78e3eeafc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.588977 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '259f668fda3d4c39a59843c78e3eeafc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.603938 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-8cd6a66f-02da-48f8-9d9a-26b7720010db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '259f668fda3d4c39a59843c78e3eeafc'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:49.615038 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1386/2810] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:49 2022] DELETE /identity/v3/users/259f668fda3d4c39a59843c78e3eeafc => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:09:49.900460 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:49.902436 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:49.917169 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38521c5-35d0-42d8-917e-f5bdfdf1b05e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:49.938170 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:49.979534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:49.980080 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:49.980353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:49.980622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] PATH_INFO: `/v3/projects/53623f8a24e34065855fa6b6e03d6b2a` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:49.981736 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53623f8a24e34065855fa6b6e03d6b2a)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:49.983739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.035358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53623f8a24e34065855fa6b6e03d6b2a', 'request_id': 'req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:50.041542 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53623f8a24e34065855fa6b6e03d6b2a', 'request_id': 'req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:50.048035 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53623f8a24e34065855fa6b6e03d6b2a', 'request_id': 'req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:50.063758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] The token cache is being invalidate because project 53623f8a24e34065855fa6b6e03d6b2a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:50.064159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:50.064819 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ac45f414-4ae3-4a13-9f8b-ba08902f25b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:50.066423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1425/2811] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:49 2022] DELETE /identity/v3/projects/53623f8a24e34065855fa6b6e03d6b2a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:50.073518 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a4a2f-e529-4110-b297-8099cecbdd96 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.074013 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a4a2f-e529-4110-b297-8099cecbdd96 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.074171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5a4a2f-e529-4110-b297-8099cecbdd96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.095724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-db5a4a2f-e529-4110-b297-8099cecbdd96 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:50.097108 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-db5a4a2f-e529-4110-b297-8099cecbdd96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:50.162982 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1387/2812] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:50.171261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4552c3-b0ee-4595-8d25-16cfc88e0532 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.171541 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4552c3-b0ee-4595-8d25-16cfc88e0532 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.171813 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4552c3-b0ee-4595-8d25-16cfc88e0532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.193392 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-ee4552c3-b0ee-4595-8d25-16cfc88e0532 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:50.194835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ee4552c3-b0ee-4595-8d25-16cfc88e0532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:50.204890 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1426/2813] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:50.209594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a4a2f-e529-4110-b297-8099cecbdd96 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.219939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5a4a2f-e529-4110-b297-8099cecbdd96 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.225841 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d26c3-3138-4b39-89f1-5aa9555ee63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.226487 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d26c3-3138-4b39-89f1-5aa9555ee63a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.226770 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d26c3-3138-4b39-89f1-5aa9555ee63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.227043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307d26c3-3138-4b39-89f1-5aa9555ee63a admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.227698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d26c3-3138-4b39-89f1-5aa9555ee63a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:09:50.228006 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d26c3-3138-4b39-89f1-5aa9555ee63a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.229713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307d26c3-3138-4b39-89f1-5aa9555ee63a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.242276 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1388/2814] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:09:50.246798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4552c3-b0ee-4595-8d25-16cfc88e0532 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.256351 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4552c3-b0ee-4595-8d25-16cfc88e0532 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.262282 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fbf6c-b86f-4752-9bc1-2df973da7d11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.262883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fbf6c-b86f-4752-9bc1-2df973da7d11 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.263183 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fbf6c-b86f-4752-9bc1-2df973da7d11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.263564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2fbf6c-b86f-4752-9bc1-2df973da7d11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.264191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fbf6c-b86f-4752-9bc1-2df973da7d11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.266125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2fbf6c-b86f-4752-9bc1-2df973da7d11 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.304341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1427/2815] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:09:50 2022] POST /identity/v3/projects => generated 387 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:50.310805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d26c3-3138-4b39-89f1-5aa9555ee63a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.320706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307d26c3-3138-4b39-89f1-5aa9555ee63a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.389974 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e59150-452d-4a00-8571-e59ecd81610b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.390691 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e59150-452d-4a00-8571-e59ecd81610b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.390966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e59150-452d-4a00-8571-e59ecd81610b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.391249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e59150-452d-4a00-8571-e59ecd81610b admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.391883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e59150-452d-4a00-8571-e59ecd81610b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.393816 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e59150-452d-4a00-8571-e59ecd81610b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.418100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1389/2816] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:09:50 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:50.422950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fbf6c-b86f-4752-9bc1-2df973da7d11 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.433859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2fbf6c-b86f-4752-9bc1-2df973da7d11 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.439149 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322c7da-6fb2-422e-936e-a0fd267fa231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.439748 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322c7da-6fb2-422e-936e-a0fd267fa231 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.440026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322c7da-6fb2-422e-936e-a0fd267fa231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.440296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322c7da-6fb2-422e-936e-a0fd267fa231 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.440934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322c7da-6fb2-422e-936e-a0fd267fa231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.442583 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322c7da-6fb2-422e-936e-a0fd267fa231 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.453294 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1428/2817] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:50.457647 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e59150-452d-4a00-8571-e59ecd81610b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.467700 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e59150-452d-4a00-8571-e59ecd81610b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.472671 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05da77-5bda-43b6-b924-12bccca4bff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.473254 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05da77-5bda-43b6-b924-12bccca4bff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.473529 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05da77-5bda-43b6-b924-12bccca4bff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.473809 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05da77-5bda-43b6-b924-12bccca4bff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.474470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05da77-5bda-43b6-b924-12bccca4bff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.476065 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a05da77-5bda-43b6-b924-12bccca4bff4 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.494994 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1390/2818] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:50.499523 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322c7da-6fb2-422e-936e-a0fd267fa231 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.510999 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322c7da-6fb2-422e-936e-a0fd267fa231 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.516930 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b536a-12e6-4b8b-b056-4a0e0a6643af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.517643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b536a-12e6-4b8b-b056-4a0e0a6643af admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.517927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b536a-12e6-4b8b-b056-4a0e0a6643af admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.518219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697b536a-12e6-4b8b-b056-4a0e0a6643af admin admin] PATH_INFO: `/v3/projects/7065358445db41f98c83e107fd15d97e/users/11c2bfb86b3947b3a0d82fe5a9f3f209/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.538825 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b536a-12e6-4b8b-b056-4a0e0a6643af admin admin] RBAC: Authorizing `identity:create_grant(project_id=7065358445db41f98c83e107fd15d97e, user_id=11c2bfb86b3947b3a0d82fe5a9f3f209, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.543241 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697b536a-12e6-4b8b-b056-4a0e0a6643af admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.560450 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1429/2819] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:50 2022] PUT /identity/v3/projects/7065358445db41f98c83e107fd15d97e/users/11c2bfb86b3947b3a0d82fe5a9f3f209/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:50.572590 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541db633-f3e8-45ef-912a-b3c5a3e471cb None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.572868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541db633-f3e8-45ef-912a-b3c5a3e471cb None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.573142 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541db633-f3e8-45ef-912a-b3c5a3e471cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.606956 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-541db633-f3e8-45ef-912a-b3c5a3e471cb None None] MFA Rules not processed for user `11c2bfb86b3947b3a0d82fe5a9f3f209`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:50.608634 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-541db633-f3e8-45ef-912a-b3c5a3e471cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:50.651444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:50.675949 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1391/2820] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:50 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:50.680866 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b536a-12e6-4b8b-b056-4a0e0a6643af admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.733365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:50.747427 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697b536a-12e6-4b8b-b056-4a0e0a6643af admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.753024 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50058711-83c4-4faa-82d0-2998baeb0b82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.753627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50058711-83c4-4faa-82d0-2998baeb0b82 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.753904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50058711-83c4-4faa-82d0-2998baeb0b82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.754207 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50058711-83c4-4faa-82d0-2998baeb0b82 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.754838 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50058711-83c4-4faa-82d0-2998baeb0b82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.756754 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50058711-83c4-4faa-82d0-2998baeb0b82 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.787084 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1430/2821] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:09:50 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:09:50.789614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541db633-f3e8-45ef-912a-b3c5a3e471cb None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.801829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541db633-f3e8-45ef-912a-b3c5a3e471cb None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.863175 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b379910-f030-4300-af84-d4c12d2a55f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.863785 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379910-f030-4300-af84-d4c12d2a55f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.864062 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379910-f030-4300-af84-d4c12d2a55f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.864336 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b379910-f030-4300-af84-d4c12d2a55f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.864973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b379910-f030-4300-af84-d4c12d2a55f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.866979 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b379910-f030-4300-af84-d4c12d2a55f6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.892726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1392/2822] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:09:50 2022] POST /identity/v3/users => generated 300 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:09:50.899486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50058711-83c4-4faa-82d0-2998baeb0b82 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.912652 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50058711-83c4-4faa-82d0-2998baeb0b82 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.918500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80386e98-ffb1-4e3d-b5e5-e7d29cd7a626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.919109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80386e98-ffb1-4e3d-b5e5-e7d29cd7a626 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.919405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80386e98-ffb1-4e3d-b5e5-e7d29cd7a626 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.920067 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80386e98-ffb1-4e3d-b5e5-e7d29cd7a626 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.920712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80386e98-ffb1-4e3d-b5e5-e7d29cd7a626 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.922366 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80386e98-ffb1-4e3d-b5e5-e7d29cd7a626 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.933590 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1431/2823] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:50.937865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b379910-f030-4300-af84-d4c12d2a55f6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:50.947790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b379910-f030-4300-af84-d4c12d2a55f6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:50.953172 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709433a7-43c3-4363-abce-90a66bcab21e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:50.953829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709433a7-43c3-4363-abce-90a66bcab21e admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:50.954109 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709433a7-43c3-4363-abce-90a66bcab21e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:50.954424 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709433a7-43c3-4363-abce-90a66bcab21e admin admin] PATH_INFO: `/v3/projects/9745b28bc59d450a883b5499879a29ee/users/5ce9c44947ae481092e21d7b880297d6/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:50.969457 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709433a7-43c3-4363-abce-90a66bcab21e admin admin] RBAC: Authorizing `identity:create_grant(project_id=9745b28bc59d450a883b5499879a29ee, user_id=5ce9c44947ae481092e21d7b880297d6, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:50.972154 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709433a7-43c3-4363-abce-90a66bcab21e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:50.987995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1393/2824] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:50 2022] PUT /identity/v3/projects/9745b28bc59d450a883b5499879a29ee/users/5ce9c44947ae481092e21d7b880297d6/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:50.992123 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80386e98-ffb1-4e3d-b5e5-e7d29cd7a626 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:51.002965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80386e98-ffb1-4e3d-b5e5-e7d29cd7a626 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:51.060784 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6d14df-2090-4d74-bce8-1644dad03e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:51.061393 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d14df-2090-4d74-bce8-1644dad03e48 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:51.061675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d14df-2090-4d74-bce8-1644dad03e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:51.061951 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6d14df-2090-4d74-bce8-1644dad03e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:51.062622 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d14df-2090-4d74-bce8-1644dad03e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:51.064226 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6d14df-2090-4d74-bce8-1644dad03e48 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:51.074074 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1432/2825] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:09:51.078494 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709433a7-43c3-4363-abce-90a66bcab21e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:51.090304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709433a7-43c3-4363-abce-90a66bcab21e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:51.096337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59dcf2-aa53-4d51-ab66-ead6a78cc9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:51.096976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59dcf2-aa53-4d51-ab66-ead6a78cc9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:51.097258 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59dcf2-aa53-4d51-ab66-ead6a78cc9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:51.097533 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa59dcf2-aa53-4d51-ab66-ead6a78cc9e6 admin admin] PATH_INFO: `/v3/projects/9745b28bc59d450a883b5499879a29ee/users/5ce9c44947ae481092e21d7b880297d6/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:51.099838 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59dcf2-aa53-4d51-ab66-ead6a78cc9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9745b28bc59d450a883b5499879a29ee, user_id=5ce9c44947ae481092e21d7b880297d6, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:51.102605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa59dcf2-aa53-4d51-ab66-ead6a78cc9e6 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:51.113656 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1394/2826] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:09:51 2022] PUT /identity/v3/projects/9745b28bc59d450a883b5499879a29ee/users/5ce9c44947ae481092e21d7b880297d6/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:51.124661 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef329e-8765-4d5f-8838-c74756d78f32 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:51.124945 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef329e-8765-4d5f-8838-c74756d78f32 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:51.125223 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bef329e-8765-4d5f-8838-c74756d78f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:51.158219 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-7bef329e-8765-4d5f-8838-c74756d78f32 None None] MFA Rules not processed for user `5ce9c44947ae481092e21d7b880297d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:51.159431 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-7bef329e-8765-4d5f-8838-c74756d78f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:51.238759 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1433/2827] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:51 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:51.250423 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59dcf2-aa53-4d51-ab66-ead6a78cc9e6 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:51.252311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-aa59dcf2-aa53-4d51-ab66-ead6a78cc9e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:51.311057 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa59dcf2-aa53-4d51-ab66-ead6a78cc9e6 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:51.315222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0078e729-5e1c-4f23-a9cd-8fa8e7183a5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:51.315831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078e729-5e1c-4f23-a9cd-8fa8e7183a5f service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:51.316112 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078e729-5e1c-4f23-a9cd-8fa8e7183a5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:51.316377 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0078e729-5e1c-4f23-a9cd-8fa8e7183a5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:51.326788 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078e729-5e1c-4f23-a9cd-8fa8e7183a5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:51.328233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0078e729-5e1c-4f23-a9cd-8fa8e7183a5f service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:51.337718 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1395/2828] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:51 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:51.928719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef329e-8765-4d5f-8838-c74756d78f32 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:51.978626 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:51.998601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bef329e-8765-4d5f-8838-c74756d78f32 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:52.004499 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:52.005140 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:52.005419 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:52.005686 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] PATH_INFO: `/v3/users/11c2bfb86b3947b3a0d82fe5a9f3f209` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:52.006937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11c2bfb86b3947b3a0d82fe5a9f3f209)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:52.008886 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:52.065859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11c2bfb86b3947b3a0d82fe5a9f3f209'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.066534 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11c2bfb86b3947b3a0d82fe5a9f3f209'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.076152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11c2bfb86b3947b3a0d82fe5a9f3f209'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.091290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11c2bfb86b3947b3a0d82fe5a9f3f209'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.092037 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11c2bfb86b3947b3a0d82fe5a9f3f209'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.105114 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1434/2829] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:51 2022] DELETE /identity/v3/users/11c2bfb86b3947b3a0d82fe5a9f3f209 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:52.110274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7eaec-3b0f-40e0-9f91-61d77ee2653d None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:52.110790 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7eaec-3b0f-40e0-9f91-61d77ee2653d None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:52.110893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c7eaec-3b0f-40e0-9f91-61d77ee2653d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:52.140026 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-65c7eaec-3b0f-40e0-9f91-61d77ee2653d None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:52.141936 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-65c7eaec-3b0f-40e0-9f91-61d77ee2653d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:52.181132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:52.215335 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1396/2830] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:09:52.225635 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:52.227281 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:52.289244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c99993-fe9a-4e38-8193-1d444eee3279 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:52.293942 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9246b1-37a9-44db-a5a0-713f6fc4256e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:52.294595 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9246b1-37a9-44db-a5a0-713f6fc4256e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:52.294876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9246b1-37a9-44db-a5a0-713f6fc4256e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:52.295132 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9246b1-37a9-44db-a5a0-713f6fc4256e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:52.304981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9246b1-37a9-44db-a5a0-713f6fc4256e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:52.306818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9246b1-37a9-44db-a5a0-713f6fc4256e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:52.316490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1435/2831] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:09:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:09:52.602002 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7eaec-3b0f-40e0-9f91-61d77ee2653d None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:52.603862 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-65c7eaec-3b0f-40e0-9f91-61d77ee2653d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:52.618227 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c7eaec-3b0f-40e0-9f91-61d77ee2653d None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:52.623565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:52.624238 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:52.624510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:52.624779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] PATH_INFO: `/v3/projects/7065358445db41f98c83e107fd15d97e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:52.626261 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7065358445db41f98c83e107fd15d97e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:52.628191 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:52.677727 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7065358445db41f98c83e107fd15d97e', 'request_id': 'req-91b5056c-3d25-4978-8bac-b872074a5b93'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.684471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7065358445db41f98c83e107fd15d97e', 'request_id': 'req-91b5056c-3d25-4978-8bac-b872074a5b93'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.691622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7065358445db41f98c83e107fd15d97e', 'request_id': 'req-91b5056c-3d25-4978-8bac-b872074a5b93'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.708007 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] The token cache is being invalidate because project 7065358445db41f98c83e107fd15d97e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:52.708304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.709029 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-91b5056c-3d25-4978-8bac-b872074a5b93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.710612 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1397/2832] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:52 2022] DELETE /identity/v3/projects/7065358445db41f98c83e107fd15d97e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:52.714917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9246b1-37a9-44db-a5a0-713f6fc4256e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:52.725418 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9246b1-37a9-44db-a5a0-713f6fc4256e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:52.784998 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:52.785638 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:52.785977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:52.786302 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] PATH_INFO: `/v3/users/5ce9c44947ae481092e21d7b880297d6` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:52.787486 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ce9c44947ae481092e21d7b880297d6)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:52.789395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:52.795015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:09:52.818617 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3b3ee-0ea1-40d2-abf1-2ca20e50c4d1 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:52.818860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3b3ee-0ea1-40d2-abf1-2ca20e50c4d1 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:52.819147 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3b3ee-0ea1-40d2-abf1-2ca20e50c4d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:52.839428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3ac3b3ee-0ea1-40d2-abf1-2ca20e50c4d1 None None] MFA Rules not processed for user `67fa873d16414749bc478a59de3ec5b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:09:52.840712 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3ac3b3ee-0ea1-40d2-abf1-2ca20e50c4d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:52.849290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ce9c44947ae481092e21d7b880297d6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.849962 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ce9c44947ae481092e21d7b880297d6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.861283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ce9c44947ae481092e21d7b880297d6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.874711 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ce9c44947ae481092e21d7b880297d6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.875564 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ce9c44947ae481092e21d7b880297d6'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:52.888600 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1436/2833] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:09:52 2022] DELETE /identity/v3/users/5ce9c44947ae481092e21d7b880297d6 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:09:52.902661 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1398/2834] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:09:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:09:53.171493 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:09:53.173780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:09:53.242395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2f0fe3-d8c0-43a2-bb90-bfee27dbc589 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:09:53.247760 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:09:53.248424 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:09:53.248702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:09:53.248978 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] PATH_INFO: `/v3/projects/9745b28bc59d450a883b5499879a29ee` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:09:53.250209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9745b28bc59d450a883b5499879a29ee)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:09:53.252438 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:09:53.307180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9745b28bc59d450a883b5499879a29ee', 'request_id': 'req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:53.314451 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9745b28bc59d450a883b5499879a29ee', 'request_id': 'req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:53.320775 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9745b28bc59d450a883b5499879a29ee', 'request_id': 'req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:53.335325 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] The token cache is being invalidate because project 9745b28bc59d450a883b5499879a29ee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:09:53.335620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:53.336287 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:09:53.337790 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1437/2835] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:09:53 2022] DELETE /identity/v3/projects/9745b28bc59d450a883b5499879a29ee => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:10:02.896160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=62,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:10:03.332135 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=74,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:10:14.048059 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4a0a-b26b-4f07-a0fe-9f816f86d80c None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:10:14.048560 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4a0a-b26b-4f07-a0fe-9f816f86d80c None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:10:14.048697 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9f4a0a-b26b-4f07-a0fe-9f816f86d80c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:10:14.069292 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-9b9f4a0a-b26b-4f07-a0fe-9f816f86d80c None None] MFA Rules not processed for user `a8576530b4ed4749a1e138f975d9b8b9`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:10:14.070548 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b9f4a0a-b26b-4f07-a0fe-9f816f86d80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:10:14.134163 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1399/2836] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:10:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:10:24.124965 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=25 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:05.818487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:05.820121 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:05.880917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf241ad-ef42-41fc-ad3c-feef3f4f7c07 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:05.884937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:05.885540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:05.885816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:05.886118 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:05.888700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:05.987108 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:05.988562 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:06.011255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1438/2837] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:15.233496 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f4a0a-b26b-4f07-a0fe-9f816f86d80c None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:15.235883 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-9b9f4a0a-b26b-4f07-a0fe-9f816f86d80c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:15.257903 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9f4a0a-b26b-4f07-a0fe-9f816f86d80c None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:15.262746 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246cedf-d9a1-4275-9ef4-c4b35b45d844 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:15.263356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246cedf-d9a1-4275-9ef4-c4b35b45d844 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:15.263633 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246cedf-d9a1-4275-9ef4-c4b35b45d844 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:15.263897 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f246cedf-d9a1-4275-9ef4-c4b35b45d844 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:15.266068 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f246cedf-d9a1-4275-9ef4-c4b35b45d844 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:15.281525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246cedf-d9a1-4275-9ef4-c4b35b45d844 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:15.283027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f246cedf-d9a1-4275-9ef4-c4b35b45d844 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:15.293954 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1400/2838] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:16.007614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=66 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:25.289327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:38.970130 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:38.971601 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:39.036651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42865b9a-3ff0-4dcf-b698-e765c778ad50 service cinder] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:39.041618 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:39.042296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.042572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.042847 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] PATH_INFO: `/v3/users/a8576530b4ed4749a1e138f975d9b8b9` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.044008 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8576530b4ed4749a1e138f975d9b8b9)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:39.045943 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:39.105399 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8576530b4ed4749a1e138f975d9b8b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.106041 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8576530b4ed4749a1e138f975d9b8b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.115737 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8576530b4ed4749a1e138f975d9b8b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.127598 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8576530b4ed4749a1e138f975d9b8b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.128174 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-b6584a91-99c3-4e2d-a7ed-93132ab9c10b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8576530b4ed4749a1e138f975d9b8b9'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.140200 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1439/2839] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:11:38 2022] DELETE /identity/v3/users/a8576530b4ed4749a1e138f975d9b8b9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:39.317393 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246cedf-d9a1-4275-9ef4-c4b35b45d844 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:39.329054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f246cedf-d9a1-4275-9ef4-c4b35b45d844 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:39.390043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:39.390858 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.391215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.391620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] PATH_INFO: `/v3/projects/0f2d19ffb32443babaa154c5766b6982` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.393539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f2d19ffb32443babaa154c5766b6982)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:39.396793 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:39.448902 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f2d19ffb32443babaa154c5766b6982', 'request_id': 'req-d7138af7-a8e1-4f73-a978-138db916cdde'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.455559 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f2d19ffb32443babaa154c5766b6982', 'request_id': 'req-d7138af7-a8e1-4f73-a978-138db916cdde'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.462943 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f2d19ffb32443babaa154c5766b6982', 'request_id': 'req-d7138af7-a8e1-4f73-a978-138db916cdde'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.477008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] The token cache is being invalidate because project 0f2d19ffb32443babaa154c5766b6982 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:11:39.477301 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.477952 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-d7138af7-a8e1-4f73-a978-138db916cdde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:39.479507 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1401/2840] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:11:39 2022] DELETE /identity/v3/projects/0f2d19ffb32443babaa154c5766b6982 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:39.484670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbba76a-d2c0-4a5f-a073-1a9a4e65330e None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.484963 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbba76a-d2c0-4a5f-a073-1a9a4e65330e None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.485245 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbba76a-d2c0-4a5f-a073-1a9a4e65330e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.505678 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-2cbba76a-d2c0-4a5f-a073-1a9a4e65330e None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:39.507489 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-2cbba76a-d2c0-4a5f-a073-1a9a4e65330e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:39.572594 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1440/2841] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:39.581631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e839a-5ba9-4a58-b005-b28d708c0080 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.581913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e839a-5ba9-4a58-b005-b28d708c0080 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.582337 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6e839a-5ba9-4a58-b005-b28d708c0080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.603527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-3a6e839a-5ba9-4a58-b005-b28d708c0080 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:39.604726 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3a6e839a-5ba9-4a58-b005-b28d708c0080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:39.613367 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1402/2842] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:39.617857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbba76a-d2c0-4a5f-a073-1a9a4e65330e None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:39.627824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbba76a-d2c0-4a5f-a073-1a9a4e65330e None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:39.632835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846f31-5e43-4762-805d-2f6e73b19822 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:39.633429 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846f31-5e43-4762-805d-2f6e73b19822 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.633702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846f31-5e43-4762-805d-2f6e73b19822 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.633968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51846f31-5e43-4762-805d-2f6e73b19822 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.634634 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846f31-5e43-4762-805d-2f6e73b19822 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:11:39.634934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846f31-5e43-4762-805d-2f6e73b19822 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:39.636524 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51846f31-5e43-4762-805d-2f6e73b19822 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:39.653832 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1441/2843] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:11:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:11:39.660569 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e839a-5ba9-4a58-b005-b28d708c0080 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:39.671287 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6e839a-5ba9-4a58-b005-b28d708c0080 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:39.677100 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0733e3c-b7cc-4d58-83d6-cc4c1e862c3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:39.677715 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0733e3c-b7cc-4d58-83d6-cc4c1e862c3c admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.677968 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0733e3c-b7cc-4d58-83d6-cc4c1e862c3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.678365 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0733e3c-b7cc-4d58-83d6-cc4c1e862c3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.678996 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0733e3c-b7cc-4d58-83d6-cc4c1e862c3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:39.681283 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0733e3c-b7cc-4d58-83d6-cc4c1e862c3c admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:39.722040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1403/2844] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:11:39 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:11:39.726623 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846f31-5e43-4762-805d-2f6e73b19822 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:39.737022 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51846f31-5e43-4762-805d-2f6e73b19822 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:39.740801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:39.797510 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce1edd-a222-4c2c-ac02-b087f9eb831e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:39.798227 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce1edd-a222-4c2c-ac02-b087f9eb831e admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.798401 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce1edd-a222-4c2c-ac02-b087f9eb831e admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.798712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce1edd-a222-4c2c-ac02-b087f9eb831e admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.799340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce1edd-a222-4c2c-ac02-b087f9eb831e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:39.801264 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ce1edd-a222-4c2c-ac02-b087f9eb831e admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:39.826276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1442/2845] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:11:39 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:11:39.830714 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0733e3c-b7cc-4d58-83d6-cc4c1e862c3c admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:39.841305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0733e3c-b7cc-4d58-83d6-cc4c1e862c3c admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:39.846904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb117b-1695-4f3b-aba6-fc8d20b2e448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:39.847521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb117b-1695-4f3b-aba6-fc8d20b2e448 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.847821 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb117b-1695-4f3b-aba6-fc8d20b2e448 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.848167 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5eb117b-1695-4f3b-aba6-fc8d20b2e448 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.848815 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb117b-1695-4f3b-aba6-fc8d20b2e448 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:39.850535 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5eb117b-1695-4f3b-aba6-fc8d20b2e448 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:39.861927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1404/2846] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:39.866742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce1edd-a222-4c2c-ac02-b087f9eb831e admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:39.877172 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce1edd-a222-4c2c-ac02-b087f9eb831e admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:39.882633 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c572507b-9ff6-4949-b7f1-afe7ff6058ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:39.883255 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572507b-9ff6-4949-b7f1-afe7ff6058ee admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.883537 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572507b-9ff6-4949-b7f1-afe7ff6058ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.883801 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c572507b-9ff6-4949-b7f1-afe7ff6058ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.884441 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572507b-9ff6-4949-b7f1-afe7ff6058ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:39.886040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c572507b-9ff6-4949-b7f1-afe7ff6058ee admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:39.896726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1443/2847] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:39.901449 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb117b-1695-4f3b-aba6-fc8d20b2e448 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:39.911581 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5eb117b-1695-4f3b-aba6-fc8d20b2e448 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:39.916997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fee9de-4432-474e-b82c-14fc44954473 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:39.917654 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fee9de-4432-474e-b82c-14fc44954473 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.917932 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fee9de-4432-474e-b82c-14fc44954473 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.918240 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fee9de-4432-474e-b82c-14fc44954473 admin admin] PATH_INFO: `/v3/projects/c256c5ce95db432b850fd7c7392ac214/users/bbebe495e6da4c80b5c8ffa30069d4e3/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.934624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fee9de-4432-474e-b82c-14fc44954473 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c256c5ce95db432b850fd7c7392ac214, user_id=bbebe495e6da4c80b5c8ffa30069d4e3, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:39.937492 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fee9de-4432-474e-b82c-14fc44954473 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:39.954169 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1405/2848] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:11:39 2022] PUT /identity/v3/projects/c256c5ce95db432b850fd7c7392ac214/users/bbebe495e6da4c80b5c8ffa30069d4e3/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:39.964727 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9da39-638e-43c5-8f79-4a18560f50ea None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:39.965015 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9da39-638e-43c5-8f79-4a18560f50ea None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:39.965317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba9da39-638e-43c5-8f79-4a18560f50ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:39.994863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-1ba9da39-638e-43c5-8f79-4a18560f50ea None None] MFA Rules not processed for user `bbebe495e6da4c80b5c8ffa30069d4e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:39.996070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1ba9da39-638e-43c5-8f79-4a18560f50ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:40.069696 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1444/2849] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:40.081528 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fee9de-4432-474e-b82c-14fc44954473 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:40.083621 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-10fee9de-4432-474e-b82c-14fc44954473 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:40.107884 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:40.166892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fee9de-4432-474e-b82c-14fc44954473 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:40.172565 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b317729-ab4d-4167-8153-5f3050085ffa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:40.173184 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b317729-ab4d-4167-8153-5f3050085ffa service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:40.173463 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b317729-ab4d-4167-8153-5f3050085ffa service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:40.173737 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b317729-ab4d-4167-8153-5f3050085ffa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:40.182346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b317729-ab4d-4167-8153-5f3050085ffa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:40.183950 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b317729-ab4d-4167-8153-5f3050085ffa service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:40.193632 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1406/2850] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:41.012927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9da39-638e-43c5-8f79-4a18560f50ea None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:41.014620 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-1ba9da39-638e-43c5-8f79-4a18560f50ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:41.093571 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba9da39-638e-43c5-8f79-4a18560f50ea None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:41.097217 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdfab330-908d-4b02-bc1e-077d0b9d1a3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:41.097839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfab330-908d-4b02-bc1e-077d0b9d1a3e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:41.098141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfab330-908d-4b02-bc1e-077d0b9d1a3e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:41.098416 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdfab330-908d-4b02-bc1e-077d0b9d1a3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:41.100221 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-cdfab330-908d-4b02-bc1e-077d0b9d1a3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:41.110397 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfab330-908d-4b02-bc1e-077d0b9d1a3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:41.111901 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdfab330-908d-4b02-bc1e-077d0b9d1a3e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:41.120641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1445/2851] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:47.566622 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b317729-ab4d-4167-8153-5f3050085ffa service nova] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:47.634350 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b317729-ab4d-4167-8153-5f3050085ffa service nova] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:47.640577 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:47.641215 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:47.641495 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:47.641763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] PATH_INFO: `/v3/users/bbebe495e6da4c80b5c8ffa30069d4e3` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:47.643135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbebe495e6da4c80b5c8ffa30069d4e3)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:47.645088 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:47.699753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbebe495e6da4c80b5c8ffa30069d4e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:47.708638 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbebe495e6da4c80b5c8ffa30069d4e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:47.722206 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbebe495e6da4c80b5c8ffa30069d4e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:47.723256 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbebe495e6da4c80b5c8ffa30069d4e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:47.734916 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbebe495e6da4c80b5c8ffa30069d4e3'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:47.744098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1407/2852] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:11:47 2022] DELETE /identity/v3/users/bbebe495e6da4c80b5c8ffa30069d4e3 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:47.748917 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66f577-bfb0-4f9f-9d2f-7ff0ea66f021 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:47.749496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66f577-bfb0-4f9f-9d2f-7ff0ea66f021 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:47.749496 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af66f577-bfb0-4f9f-9d2f-7ff0ea66f021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:47.769586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-af66f577-bfb0-4f9f-9d2f-7ff0ea66f021 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:47.771026 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-af66f577-bfb0-4f9f-9d2f-7ff0ea66f021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:47.784906 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:47.836611 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1446/2853] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:47.849411 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:47.850893 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:47.878102 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:47.911325 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3951fe3f-9f19-4406-aa84-879c253db27e admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:47.915505 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6567b5-bbd0-4f55-a314-518e7491a124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:47.916155 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6567b5-bbd0-4f55-a314-518e7491a124 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:47.916426 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6567b5-bbd0-4f55-a314-518e7491a124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:47.916705 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6567b5-bbd0-4f55-a314-518e7491a124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:47.926097 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6567b5-bbd0-4f55-a314-518e7491a124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:47.927537 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6567b5-bbd0-4f55-a314-518e7491a124 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:47.937387 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1408/2854] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:48.238014 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66f577-bfb0-4f9f-9d2f-7ff0ea66f021 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:48.239627 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-af66f577-bfb0-4f9f-9d2f-7ff0ea66f021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:48.255796 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af66f577-bfb0-4f9f-9d2f-7ff0ea66f021 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:48.270684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:48.271345 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.271624 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.271888 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] PATH_INFO: `/v3/projects/c256c5ce95db432b850fd7c7392ac214` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.273202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] RBAC: Authorizing `identity:delete_project(project_id=c256c5ce95db432b850fd7c7392ac214)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:48.275539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:48.327152 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c256c5ce95db432b850fd7c7392ac214', 'request_id': 'req-31a6f056-f592-4e86-871b-768d8fce40de'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:48.333535 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c256c5ce95db432b850fd7c7392ac214', 'request_id': 'req-31a6f056-f592-4e86-871b-768d8fce40de'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:48.339490 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c256c5ce95db432b850fd7c7392ac214', 'request_id': 'req-31a6f056-f592-4e86-871b-768d8fce40de'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:48.354248 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] The token cache is being invalidate because project c256c5ce95db432b850fd7c7392ac214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:11:48.354539 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:48.355160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-31a6f056-f592-4e86-871b-768d8fce40de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:48.356658 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1447/2855] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:11:48 2022] DELETE /identity/v3/projects/c256c5ce95db432b850fd7c7392ac214 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:48.361660 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc373d73-ba3f-4da7-8350-b2d2e3d8555b None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.361953 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc373d73-ba3f-4da7-8350-b2d2e3d8555b None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.362266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc373d73-ba3f-4da7-8350-b2d2e3d8555b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.383521 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-cc373d73-ba3f-4da7-8350-b2d2e3d8555b None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:48.384859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-cc373d73-ba3f-4da7-8350-b2d2e3d8555b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:48.455132 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1409/2856] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:48.464781 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d7a2d-7efd-42ba-8e03-36ccb6b02c20 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.465161 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d7a2d-7efd-42ba-8e03-36ccb6b02c20 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.465318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763d7a2d-7efd-42ba-8e03-36ccb6b02c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.492402 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-763d7a2d-7efd-42ba-8e03-36ccb6b02c20 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:48.494113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-763d7a2d-7efd-42ba-8e03-36ccb6b02c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:48.502870 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1448/2857] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:48.507736 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc373d73-ba3f-4da7-8350-b2d2e3d8555b None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:48.522248 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc373d73-ba3f-4da7-8350-b2d2e3d8555b None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:48.529846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ce5c1-4ae8-4571-bb12-686c3628ca5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:48.530500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ce5c1-4ae8-4571-bb12-686c3628ca5f admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.530908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ce5c1-4ae8-4571-bb12-686c3628ca5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.531027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52ce5c1-4ae8-4571-bb12-686c3628ca5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.531694 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ce5c1-4ae8-4571-bb12-686c3628ca5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:11:48.531998 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ce5c1-4ae8-4571-bb12-686c3628ca5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:48.533623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52ce5c1-4ae8-4571-bb12-686c3628ca5f admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:48.546356 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1410/2858] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:11:48.553368 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d7a2d-7efd-42ba-8e03-36ccb6b02c20 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:48.567159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763d7a2d-7efd-42ba-8e03-36ccb6b02c20 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:48.574447 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae073c59-a8dd-486c-a1e3-fdfd86068293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:48.575117 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae073c59-a8dd-486c-a1e3-fdfd86068293 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.575395 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae073c59-a8dd-486c-a1e3-fdfd86068293 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.575663 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae073c59-a8dd-486c-a1e3-fdfd86068293 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.576298 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae073c59-a8dd-486c-a1e3-fdfd86068293 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:48.578443 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae073c59-a8dd-486c-a1e3-fdfd86068293 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:48.611307 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1449/2859] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:11:48 2022] POST /identity/v3/projects => generated 371 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:11:48.615171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ce5c1-4ae8-4571-bb12-686c3628ca5f admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:48.627575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52ce5c1-4ae8-4571-bb12-686c3628ca5f admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:48.693263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7159f8b-68c7-49cc-9137-e190ae949347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:48.693892 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7159f8b-68c7-49cc-9137-e190ae949347 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.694274 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7159f8b-68c7-49cc-9137-e190ae949347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.694555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7159f8b-68c7-49cc-9137-e190ae949347 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.695186 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7159f8b-68c7-49cc-9137-e190ae949347 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:48.697233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7159f8b-68c7-49cc-9137-e190ae949347 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:48.729798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1411/2860] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:11:48 2022] POST /identity/v3/users => generated 286 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:11:48.734299 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae073c59-a8dd-486c-a1e3-fdfd86068293 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:48.748482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae073c59-a8dd-486c-a1e3-fdfd86068293 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:48.755120 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5000ef-0329-4650-b3e0-1850c416ad43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:48.755758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5000ef-0329-4650-b3e0-1850c416ad43 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.756135 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5000ef-0329-4650-b3e0-1850c416ad43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.756474 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5000ef-0329-4650-b3e0-1850c416ad43 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.757194 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5000ef-0329-4650-b3e0-1850c416ad43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:48.759198 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5000ef-0329-4650-b3e0-1850c416ad43 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:48.772317 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1450/2861] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:48.778614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7159f8b-68c7-49cc-9137-e190ae949347 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:48.790722 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7159f8b-68c7-49cc-9137-e190ae949347 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:48.796349 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1cc68a2-ed2b-463b-9e36-58f2c3dfc1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:48.796976 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc68a2-ed2b-463b-9e36-58f2c3dfc1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.797281 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc68a2-ed2b-463b-9e36-58f2c3dfc1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.797551 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1cc68a2-ed2b-463b-9e36-58f2c3dfc1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.798241 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc68a2-ed2b-463b-9e36-58f2c3dfc1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:48.799865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1cc68a2-ed2b-463b-9e36-58f2c3dfc1b9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:48.811605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1412/2862] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:48.816643 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5000ef-0329-4650-b3e0-1850c416ad43 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:48.829129 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5000ef-0329-4650-b3e0-1850c416ad43 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:48.834875 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b0cd6-7e53-49f3-b84f-debd4e30879b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:48.835536 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b0cd6-7e53-49f3-b84f-debd4e30879b admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.835818 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b0cd6-7e53-49f3-b84f-debd4e30879b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.836073 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05b0cd6-7e53-49f3-b84f-debd4e30879b admin admin] PATH_INFO: `/v3/projects/e6c99b17634d4c1394fbe1ef73d32476/users/7828799c772a429b9f3cfdb636d2bdb9/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.851540 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b0cd6-7e53-49f3-b84f-debd4e30879b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6c99b17634d4c1394fbe1ef73d32476, user_id=7828799c772a429b9f3cfdb636d2bdb9, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:48.854487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05b0cd6-7e53-49f3-b84f-debd4e30879b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:48.870030 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1451/2863] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:11:48 2022] PUT /identity/v3/projects/e6c99b17634d4c1394fbe1ef73d32476/users/7828799c772a429b9f3cfdb636d2bdb9/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:11:48.880474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deab32-a4f7-4460-936f-bcad29502146 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:48.880763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deab32-a4f7-4460-936f-bcad29502146 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:48.881049 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58deab32-a4f7-4460-936f-bcad29502146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:48.909354 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-58deab32-a4f7-4460-936f-bcad29502146 None None] MFA Rules not processed for user `7828799c772a429b9f3cfdb636d2bdb9`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:48.910628 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-58deab32-a4f7-4460-936f-bcad29502146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:48.931448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:48.978951 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1413/2864] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:48.991218 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b0cd6-7e53-49f3-b84f-debd4e30879b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:48.993160 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-a05b0cd6-7e53-49f3-b84f-debd4e30879b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:49.036873 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:49.071788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05b0cd6-7e53-49f3-b84f-debd4e30879b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:49.078662 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e171e565-13cb-4846-a411-43721458f1c0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:49.079275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171e565-13cb-4846-a411-43721458f1c0 service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:49.079552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171e565-13cb-4846-a411-43721458f1c0 service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:49.079822 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e171e565-13cb-4846-a411-43721458f1c0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:49.088516 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171e565-13cb-4846-a411-43721458f1c0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:49.090113 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e171e565-13cb-4846-a411-43721458f1c0 service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:49.099391 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1452/2865] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:51.835566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58deab32-a4f7-4460-936f-bcad29502146 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:51.898524 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58deab32-a4f7-4460-936f-bcad29502146 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:51.903937 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:51.904572 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:51.904854 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:51.905130 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] PATH_INFO: `/v3/users/7828799c772a429b9f3cfdb636d2bdb9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:51.906319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7828799c772a429b9f3cfdb636d2bdb9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:51.908249 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:51.974027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7828799c772a429b9f3cfdb636d2bdb9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:51.983631 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7828799c772a429b9f3cfdb636d2bdb9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:51.997221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7828799c772a429b9f3cfdb636d2bdb9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:51.997919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7828799c772a429b9f3cfdb636d2bdb9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:52.004995 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7828799c772a429b9f3cfdb636d2bdb9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:52.011034 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1414/2866] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:11:51 2022] DELETE /identity/v3/users/7828799c772a429b9f3cfdb636d2bdb9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:52.015695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e69313-460f-47e8-8cde-b9b0c41b01f7 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:52.016232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e69313-460f-47e8-8cde-b9b0c41b01f7 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:52.016232 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e69313-460f-47e8-8cde-b9b0c41b01f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:52.036011 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-21e69313-460f-47e8-8cde-b9b0c41b01f7 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:52.037208 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-21e69313-460f-47e8-8cde-b9b0c41b01f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:52.102500 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1453/2867] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:52.112859 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:52.114326 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:52.163050 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:52.180245 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221f2a3-1169-4032-9607-1e80b9b68e13 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:52.184594 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593e8952-4304-42e3-bf40-fb7c6ba88d8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:52.185221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e8952-4304-42e3-bf40-fb7c6ba88d8f service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:52.185491 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e8952-4304-42e3-bf40-fb7c6ba88d8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:52.185768 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593e8952-4304-42e3-bf40-fb7c6ba88d8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:52.196915 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e8952-4304-42e3-bf40-fb7c6ba88d8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:52.198471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-593e8952-4304-42e3-bf40-fb7c6ba88d8f service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:52.208508 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1415/2868] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:52.485599 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e69313-460f-47e8-8cde-b9b0c41b01f7 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:52.487341 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-21e69313-460f-47e8-8cde-b9b0c41b01f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:52.501258 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e69313-460f-47e8-8cde-b9b0c41b01f7 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:52.506641 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:52.507340 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:52.507685 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:52.508046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] PATH_INFO: `/v3/projects/e6c99b17634d4c1394fbe1ef73d32476` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:52.509445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6c99b17634d4c1394fbe1ef73d32476)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:52.511608 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:52.566671 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6c99b17634d4c1394fbe1ef73d32476', 'request_id': 'req-2f668de1-7e10-4e82-9db8-455fb17135ce'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:52.573388 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6c99b17634d4c1394fbe1ef73d32476', 'request_id': 'req-2f668de1-7e10-4e82-9db8-455fb17135ce'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:52.579911 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6c99b17634d4c1394fbe1ef73d32476', 'request_id': 'req-2f668de1-7e10-4e82-9db8-455fb17135ce'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:52.600039 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] The token cache is being invalidate because project e6c99b17634d4c1394fbe1ef73d32476 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:11:52.600332 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:52.600994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-2f668de1-7e10-4e82-9db8-455fb17135ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:52.602604 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1454/2869] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:11:52 2022] DELETE /identity/v3/projects/e6c99b17634d4c1394fbe1ef73d32476 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:52.607876 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca507c57-962b-42b5-8f67-914223b2afca None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:52.608296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca507c57-962b-42b5-8f67-914223b2afca None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:52.608639 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca507c57-962b-42b5-8f67-914223b2afca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:52.628988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-ca507c57-962b-42b5-8f67-914223b2afca None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:52.630247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-ca507c57-962b-42b5-8f67-914223b2afca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:52.695451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1416/2870] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:11:52.703883 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676913ed-6932-4f28-b71d-91d381006981 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:52.704262 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676913ed-6932-4f28-b71d-91d381006981 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:52.704640 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676913ed-6932-4f28-b71d-91d381006981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:52.728134 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-676913ed-6932-4f28-b71d-91d381006981 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:52.729835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-676913ed-6932-4f28-b71d-91d381006981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:52.738904 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1455/2871] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:52.743539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca507c57-962b-42b5-8f67-914223b2afca None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:52.754861 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca507c57-962b-42b5-8f67-914223b2afca None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:52.759614 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8562-4609-4e18-8375-ce90027124b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:52.760361 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8562-4609-4e18-8375-ce90027124b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:52.760596 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8562-4609-4e18-8375-ce90027124b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:52.760831 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768d8562-4609-4e18-8375-ce90027124b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:52.761399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d8562-4609-4e18-8375-ce90027124b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81499) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:11:52.761711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d8562-4609-4e18-8375-ce90027124b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:52.763511 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768d8562-4609-4e18-8375-ce90027124b5 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:52.777247 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1417/2872] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:11:52.781826 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676913ed-6932-4f28-b71d-91d381006981 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:52.791596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-676913ed-6932-4f28-b71d-91d381006981 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:52.796482 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee1591-e1b0-43c4-9eed-c416c4a238d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:52.797081 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee1591-e1b0-43c4-9eed-c416c4a238d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:52.797353 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee1591-e1b0-43c4-9eed-c416c4a238d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:52.797614 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ee1591-e1b0-43c4-9eed-c416c4a238d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:52.798509 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee1591-e1b0-43c4-9eed-c416c4a238d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:52.800209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ee1591-e1b0-43c4-9eed-c416c4a238d6 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:52.848168 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1456/2873] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:11:52 2022] POST /identity/v3/projects => generated 363 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:11:52.849451 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8562-4609-4e18-8375-ce90027124b5 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:52.860597 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768d8562-4609-4e18-8375-ce90027124b5 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:52.922846 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1730d-6693-4ff9-a881-b32a52ca3655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:52.923372 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1730d-6693-4ff9-a881-b32a52ca3655 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:52.923658 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1730d-6693-4ff9-a881-b32a52ca3655 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:52.923919 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1730d-6693-4ff9-a881-b32a52ca3655 admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:52.924605 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1730d-6693-4ff9-a881-b32a52ca3655 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:52.926575 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1730d-6693-4ff9-a881-b32a52ca3655 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:52.956471 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1418/2874] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:11:52 2022] POST /identity/v3/users => generated 282 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:11:52.957985 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee1591-e1b0-43c4-9eed-c416c4a238d6 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:52.969141 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ee1591-e1b0-43c4-9eed-c416c4a238d6 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:52.974357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bfba9-6ecf-4078-905a-9e04353f5a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:52.974990 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bfba9-6ecf-4078-905a-9e04353f5a17 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:52.975270 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bfba9-6ecf-4078-905a-9e04353f5a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:52.975679 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424bfba9-6ecf-4078-905a-9e04353f5a17 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:52.976201 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bfba9-6ecf-4078-905a-9e04353f5a17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:52.978138 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424bfba9-6ecf-4078-905a-9e04353f5a17 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:52.993365 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1457/2875] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:52.996803 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1730d-6693-4ff9-a881-b32a52ca3655 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:53.015808 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1730d-6693-4ff9-a881-b32a52ca3655 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:53.028344 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f770446-ca16-464e-a24e-d5e4a55f6f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:53.029202 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f770446-ca16-464e-a24e-d5e4a55f6f4a admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.029512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f770446-ca16-464e-a24e-d5e4a55f6f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.030027 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f770446-ca16-464e-a24e-d5e4a55f6f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.030704 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f770446-ca16-464e-a24e-d5e4a55f6f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:53.032441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f770446-ca16-464e-a24e-d5e4a55f6f4a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:53.045105 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1419/2876] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:52 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:53.051249 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bfba9-6ecf-4078-905a-9e04353f5a17 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:53.066894 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424bfba9-6ecf-4078-905a-9e04353f5a17 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:53.072947 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483ff70-9851-4f53-a8f5-48ace049b01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:53.073859 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483ff70-9851-4f53-a8f5-48ace049b01c admin admin] REQUEST_METHOD: `PUT` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.074165 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483ff70-9851-4f53-a8f5-48ace049b01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.074434 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483ff70-9851-4f53-a8f5-48ace049b01c admin admin] PATH_INFO: `/v3/projects/f4f0f934dbbc4baab6a528e80a569b29/users/1415cfd2febf453dbaf9ef31e99d4e77/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.077653 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:53.094952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483ff70-9851-4f53-a8f5-48ace049b01c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f0f934dbbc4baab6a528e80a569b29, user_id=1415cfd2febf453dbaf9ef31e99d4e77, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:53.097927 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483ff70-9851-4f53-a8f5-48ace049b01c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:53.115407 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1458/2877] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:11:53 2022] PUT /identity/v3/projects/f4f0f934dbbc4baab6a528e80a569b29/users/1415cfd2febf453dbaf9ef31e99d4e77/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:53.127107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f6ed-2725-4cc3-ad94-043981fd9b38 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.127448 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f6ed-2725-4cc3-ad94-043981fd9b38 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.127763 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a5f6ed-2725-4cc3-ad94-043981fd9b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.157663 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-07a5f6ed-2725-4cc3-ad94-043981fd9b38 None None] MFA Rules not processed for user `1415cfd2febf453dbaf9ef31e99d4e77`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:53.158929 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-07a5f6ed-2725-4cc3-ad94-043981fd9b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:53.200308 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:53.229835 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1420/2878] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:53.234503 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483ff70-9851-4f53-a8f5-48ace049b01c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:53.301326 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483ff70-9851-4f53-a8f5-48ace049b01c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:53.307110 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312c563-c3b3-40ed-8cdf-a379ff6ade97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:53.307724 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312c563-c3b3-40ed-8cdf-a379ff6ade97 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.307977 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312c563-c3b3-40ed-8cdf-a379ff6ade97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.308254 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d312c563-c3b3-40ed-8cdf-a379ff6ade97 admin admin] PATH_INFO: `/v3/projects` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.308878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312c563-c3b3-40ed-8cdf-a379ff6ade97 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:53.311433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d312c563-c3b3-40ed-8cdf-a379ff6ade97 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:53.346864 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1459/2879] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:11:53 2022] POST /identity/v3/projects => generated 365 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:11:53.351264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f6ed-2725-4cc3-ad94-043981fd9b38 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:53.362745 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a5f6ed-2725-4cc3-ad94-043981fd9b38 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:53.428618 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eafd3-4cf7-437b-9c43-c2f7b39699db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:53.429232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eafd3-4cf7-437b-9c43-c2f7b39699db admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.429516 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eafd3-4cf7-437b-9c43-c2f7b39699db admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.429762 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2eafd3-4cf7-437b-9c43-c2f7b39699db admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.430805 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2eafd3-4cf7-437b-9c43-c2f7b39699db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:53.432891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2eafd3-4cf7-437b-9c43-c2f7b39699db admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:53.461510 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1421/2880] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:11:53 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 13 07:11:53.462899 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312c563-c3b3-40ed-8cdf-a379ff6ade97 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:53.473405 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d312c563-c3b3-40ed-8cdf-a379ff6ade97 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:53.478700 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79608f39-2d29-464d-8a8f-194c7a1fbd4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:53.479463 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79608f39-2d29-464d-8a8f-194c7a1fbd4b admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.479791 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79608f39-2d29-464d-8a8f-194c7a1fbd4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.480133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79608f39-2d29-464d-8a8f-194c7a1fbd4b admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.480850 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79608f39-2d29-464d-8a8f-194c7a1fbd4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:53.482555 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79608f39-2d29-464d-8a8f-194c7a1fbd4b admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:53.494527 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1460/2881] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:53.497820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eafd3-4cf7-437b-9c43-c2f7b39699db admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:53.508131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2eafd3-4cf7-437b-9c43-c2f7b39699db admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:53.513539 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a3dc8-5672-4926-9292-ef9cbef198b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:53.514264 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a3dc8-5672-4926-9292-ef9cbef198b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.514547 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a3dc8-5672-4926-9292-ef9cbef198b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.514820 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12a3dc8-5672-4926-9292-ef9cbef198b1 admin admin] PATH_INFO: `/v3/projects/b0890db7492e4dda8ed53da8de84283a/users/466885a0c92440dfa96ec31b16f0650d/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.530602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a3dc8-5672-4926-9292-ef9cbef198b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0890db7492e4dda8ed53da8de84283a, user_id=466885a0c92440dfa96ec31b16f0650d, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:53.533266 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12a3dc8-5672-4926-9292-ef9cbef198b1 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:53.550001 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1422/2882] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:11:53 2022] PUT /identity/v3/projects/b0890db7492e4dda8ed53da8de84283a/users/466885a0c92440dfa96ec31b16f0650d/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:53.555310 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79608f39-2d29-464d-8a8f-194c7a1fbd4b admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:53.566357 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79608f39-2d29-464d-8a8f-194c7a1fbd4b admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:53.634187 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15d96b6-6e73-4620-b48c-570fe727b01d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:53.634869 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d96b6-6e73-4620-b48c-570fe727b01d admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.635220 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d96b6-6e73-4620-b48c-570fe727b01d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.635425 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15d96b6-6e73-4620-b48c-570fe727b01d admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.636070 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d96b6-6e73-4620-b48c-570fe727b01d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:53.637695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15d96b6-6e73-4620-b48c-570fe727b01d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:53.648159 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1461/2883] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:53 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:53.652752 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a3dc8-5672-4926-9292-ef9cbef198b1 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:53.663771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12a3dc8-5672-4926-9292-ef9cbef198b1 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:53.668872 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742cf8d-13cc-4925-a6c0-7a3e2267ac24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:53.669530 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742cf8d-13cc-4925-a6c0-7a3e2267ac24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.669811 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742cf8d-13cc-4925-a6c0-7a3e2267ac24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.670110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b742cf8d-13cc-4925-a6c0-7a3e2267ac24 admin admin] PATH_INFO: `/v3/projects/b0890db7492e4dda8ed53da8de84283a/users/466885a0c92440dfa96ec31b16f0650d/roles/756f3eabecc842a0aec6d2e8a9ce5de9` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.672124 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742cf8d-13cc-4925-a6c0-7a3e2267ac24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0890db7492e4dda8ed53da8de84283a, user_id=466885a0c92440dfa96ec31b16f0650d, role_id=756f3eabecc842a0aec6d2e8a9ce5de9)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:53.674837 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b742cf8d-13cc-4925-a6c0-7a3e2267ac24 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:53.686297 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1423/2884] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:11:53 2022] PUT /identity/v3/projects/b0890db7492e4dda8ed53da8de84283a/users/466885a0c92440dfa96ec31b16f0650d/roles/756f3eabecc842a0aec6d2e8a9ce5de9 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:53.696647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786b6f2-1d9f-4509-9ef8-db5a6ac953b6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.696929 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786b6f2-1d9f-4509-9ef8-db5a6ac953b6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.697177 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f786b6f2-1d9f-4509-9ef8-db5a6ac953b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.727567 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f786b6f2-1d9f-4509-9ef8-db5a6ac953b6 None None] MFA Rules not processed for user `466885a0c92440dfa96ec31b16f0650d`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:53.729358 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f786b6f2-1d9f-4509-9ef8-db5a6ac953b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:53.776972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:53.809111 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1462/2885] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:53.820515 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742cf8d-13cc-4925-a6c0-7a3e2267ac24 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:53.822434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-b742cf8d-13cc-4925-a6c0-7a3e2267ac24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:53.889216 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b742cf8d-13cc-4925-a6c0-7a3e2267ac24 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:53.893171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a396d4e-4e0c-43c8-961d-7d9ec97e8c23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:53.893794 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396d4e-4e0c-43c8-961d-7d9ec97e8c23 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:53.894071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396d4e-4e0c-43c8-961d-7d9ec97e8c23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:53.894379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a396d4e-4e0c-43c8-961d-7d9ec97e8c23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:53.904311 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a396d4e-4e0c-43c8-961d-7d9ec97e8c23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:53.905792 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a396d4e-4e0c-43c8-961d-7d9ec97e8c23 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:53.915230 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1424/2886] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:54.369919 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786b6f2-1d9f-4509-9ef8-db5a6ac953b6 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:54.456127 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f786b6f2-1d9f-4509-9ef8-db5a6ac953b6 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:54.461309 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:54.461950 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:54.462261 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:54.462596 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] PATH_INFO: `/v3/users/1415cfd2febf453dbaf9ef31e99d4e77` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:54.463747 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1415cfd2febf453dbaf9ef31e99d4e77)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:54.465684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:54.524049 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1415cfd2febf453dbaf9ef31e99d4e77'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:54.524863 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1415cfd2febf453dbaf9ef31e99d4e77'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:54.535528 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1415cfd2febf453dbaf9ef31e99d4e77'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:54.554586 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1415cfd2febf453dbaf9ef31e99d4e77'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:54.555318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1415cfd2febf453dbaf9ef31e99d4e77'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:54.568376 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1463/2887] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:11:54 2022] DELETE /identity/v3/users/1415cfd2febf453dbaf9ef31e99d4e77 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:11:54.572997 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f6c74-5ec5-4612-82c2-065b4e5e27f2 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:54.573415 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f6c74-5ec5-4612-82c2-065b4e5e27f2 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:54.573698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79f6c74-5ec5-4612-82c2-065b4e5e27f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:54.595444 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-c79f6c74-5ec5-4612-82c2-065b4e5e27f2 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:54.596713 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c79f6c74-5ec5-4612-82c2-065b4e5e27f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:54.633131 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:54.668035 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1425/2888] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:54.677771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:54.679290 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:54.740742 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5150218f-3691-4f25-8648-ebcb152350a9 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:54.745446 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3865d0-8069-49c3-98de-65e193f525bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:54.746433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3865d0-8069-49c3-98de-65e193f525bb service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:54.746433 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3865d0-8069-49c3-98de-65e193f525bb service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:54.746675 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3865d0-8069-49c3-98de-65e193f525bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:54.757139 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3865d0-8069-49c3-98de-65e193f525bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:54.758625 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3865d0-8069-49c3-98de-65e193f525bb service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:54.763934 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:54.769253 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1464/2889] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:55.059742 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f6c74-5ec5-4612-82c2-065b4e5e27f2 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:55.061623 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-c79f6c74-5ec5-4612-82c2-065b4e5e27f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:55.073895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79f6c74-5ec5-4612-82c2-065b4e5e27f2 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:55.078684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:55.079331 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:55.079643 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:55.079891 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] PATH_INFO: `/v3/projects/f4f0f934dbbc4baab6a528e80a569b29` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:55.081236 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f0f934dbbc4baab6a528e80a569b29)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:55.083243 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:55.134685 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f0f934dbbc4baab6a528e80a569b29', 'request_id': 'req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.141214 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f0f934dbbc4baab6a528e80a569b29', 'request_id': 'req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.147179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f0f934dbbc4baab6a528e80a569b29', 'request_id': 'req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.165043 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] The token cache is being invalidate because project f4f0f934dbbc4baab6a528e80a569b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:11:55.165428 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.166164 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.168158 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1426/2890] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:11:55 2022] DELETE /identity/v3/projects/f4f0f934dbbc4baab6a528e80a569b29 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:55.172308 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3865d0-8069-49c3-98de-65e193f525bb service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:55.183684 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3865d0-8069-49c3-98de-65e193f525bb service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:55.254651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:55.255176 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:55.255449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:55.255718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] PATH_INFO: `/v3/users/466885a0c92440dfa96ec31b16f0650d` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:55.256925 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] RBAC: Authorizing `identity:delete_user(user_id=466885a0c92440dfa96ec31b16f0650d)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:55.258878 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:55.311191 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '466885a0c92440dfa96ec31b16f0650d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.311874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '466885a0c92440dfa96ec31b16f0650d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.322417 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '466885a0c92440dfa96ec31b16f0650d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.334968 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '466885a0c92440dfa96ec31b16f0650d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.335695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-95cda997-d1d5-4c9b-b3af-ef2fe0abb989 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '466885a0c92440dfa96ec31b16f0650d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.348849 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1465/2891] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:11:55 2022] DELETE /identity/v3/users/466885a0c92440dfa96ec31b16f0650d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:55.624096 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:55.625798 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:55.639069 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da6b0da-f0b2-47b9-ad3d-cb987073bab9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:55.697368 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:55.697882 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:55.698195 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:55.698474 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] PATH_INFO: `/v3/projects/b0890db7492e4dda8ed53da8de84283a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:55.700145 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0890db7492e4dda8ed53da8de84283a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:55.701566 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:55.754773 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0890db7492e4dda8ed53da8de84283a', 'request_id': 'req-7ab0552b-069a-40c5-af55-4465b425bc67'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.761040 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0890db7492e4dda8ed53da8de84283a', 'request_id': 'req-7ab0552b-069a-40c5-af55-4465b425bc67'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.765358 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:55.767983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0890db7492e4dda8ed53da8de84283a', 'request_id': 'req-7ab0552b-069a-40c5-af55-4465b425bc67'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.783999 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] The token cache is being invalidate because project b0890db7492e4dda8ed53da8de84283a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:11:55.784294 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.784966 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-7ab0552b-069a-40c5-af55-4465b425bc67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:11:55.786602 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1427/2892] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:11:55 2022] DELETE /identity/v3/projects/b0890db7492e4dda8ed53da8de84283a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 13 07:11:55.793403 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5160570-d0c1-47b9-ab40-1da74bd8d8cb None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:55.793980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5160570-d0c1-47b9-ab40-1da74bd8d8cb None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:55.793980 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5160570-d0c1-47b9-ab40-1da74bd8d8cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:55.821180 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-f5160570-d0c1-47b9-ab40-1da74bd8d8cb None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:55.822459 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-f5160570-d0c1-47b9-ab40-1da74bd8d8cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:55.883286 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1466/2893] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:55.891420 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae0bcb-ff22-483e-8ef9-5860f437a6ac None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:55.891710 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae0bcb-ff22-483e-8ef9-5860f437a6ac None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:55.891986 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae0bcb-ff22-483e-8ef9-5860f437a6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:55.913497 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-f8ae0bcb-ff22-483e-8ef9-5860f437a6ac None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:55.914769 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-f8ae0bcb-ff22-483e-8ef9-5860f437a6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:55.924055 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1428/2894] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:55.928521 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5160570-d0c1-47b9-ab40-1da74bd8d8cb None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:55.940337 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5160570-d0c1-47b9-ab40-1da74bd8d8cb None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:55.946755 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:55.947046 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:55.947432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:55.947734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:55.948432 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:11:55.948733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:55.950577 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:55.965274 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1467/2895] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 13 07:11:55.971098 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae0bcb-ff22-483e-8ef9-5860f437a6ac None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:55.990724 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae0bcb-ff22-483e-8ef9-5860f437a6ac None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:55.995179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cff94c-ed4d-4eb6-97e4-f1b01e7b5484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:55.996110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cff94c-ed4d-4eb6-97e4-f1b01e7b5484 admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:55.996110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cff94c-ed4d-4eb6-97e4-f1b01e7b5484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:55.996110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cff94c-ed4d-4eb6-97e4-f1b01e7b5484 admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:55.997178 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cff94c-ed4d-4eb6-97e4-f1b01e7b5484 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:55.999193 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cff94c-ed4d-4eb6-97e4-f1b01e7b5484 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:56.012189 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:56.013994 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:56.023792 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:56.034458 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1429/2896] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:11:55 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:11:56.036522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cff94c-ed4d-4eb6-97e4-f1b01e7b5484 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:56.078773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb1061f-1be5-4b52-b8e0-b977b6e451ff admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:56.106908 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cff94c-ed4d-4eb6-97e4-f1b01e7b5484 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:56.111624 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d105d68-7884-4dcc-ad4d-d6e3c5211cbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:56.112235 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d105d68-7884-4dcc-ad4d-d6e3c5211cbc admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:56.112512 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d105d68-7884-4dcc-ad4d-d6e3c5211cbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:56.112810 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d105d68-7884-4dcc-ad4d-d6e3c5211cbc admin admin] PATH_INFO: `/v3/users` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:56.113442 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d105d68-7884-4dcc-ad4d-d6e3c5211cbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:56.115396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d105d68-7884-4dcc-ad4d-d6e3c5211cbc admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:56.127423 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcb704-89c5-4c79-9343-81a07e14db4e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:56.128036 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcb704-89c5-4c79-9343-81a07e14db4e service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:56.128480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcb704-89c5-4c79-9343-81a07e14db4e service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:56.128657 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bcb704-89c5-4c79-9343-81a07e14db4e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:56.130591 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-28bcb704-89c5-4c79-9343-81a07e14db4e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:56.141137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1430/2897] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:11:56 2022] POST /identity/v3/users => generated 292 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:11:56.142575 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcb704-89c5-4c79-9343-81a07e14db4e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:56.144002 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bcb704-89c5-4c79-9343-81a07e14db4e service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:56.145380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d105d68-7884-4dcc-ad4d-d6e3c5211cbc admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:56.154573 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1468/2898] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 13 07:11:56.156409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d105d68-7884-4dcc-ad4d-d6e3c5211cbc admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:56.161483 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badf98f-4cb0-492d-a6b5-565c071126cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:56.162104 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badf98f-4cb0-492d-a6b5-565c071126cd admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:56.162399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badf98f-4cb0-492d-a6b5-565c071126cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:56.162673 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5badf98f-4cb0-492d-a6b5-565c071126cd admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:56.163363 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badf98f-4cb0-492d-a6b5-565c071126cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:56.164983 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5badf98f-4cb0-492d-a6b5-565c071126cd admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:56.176028 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1431/2899] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:56.180758 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcb704-89c5-4c79-9343-81a07e14db4e service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:56.190970 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bcb704-89c5-4c79-9343-81a07e14db4e service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:56.195874 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3de3b0-fdfb-4b9d-b6b9-6c95f72b98d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:56.197086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3de3b0-fdfb-4b9d-b6b9-6c95f72b98d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:56.197086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3de3b0-fdfb-4b9d-b6b9-6c95f72b98d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:56.197086 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3de3b0-fdfb-4b9d-b6b9-6c95f72b98d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:56.197935 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3de3b0-fdfb-4b9d-b6b9-6c95f72b98d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:56.199585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3de3b0-fdfb-4b9d-b6b9-6c95f72b98d1 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:56.209902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1469/2900] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:11:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:11:56.214330 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badf98f-4cb0-492d-a6b5-565c071126cd admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:56.224800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5badf98f-4cb0-492d-a6b5-565c071126cd admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:56.230856 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccba29-feda-453f-b1ac-b824b99e29d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:56.231527 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccba29-feda-453f-b1ac-b824b99e29d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:56.231789 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccba29-feda-453f-b1ac-b824b99e29d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:56.232060 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ccba29-feda-453f-b1ac-b824b99e29d9 admin admin] PATH_INFO: `/v3/projects/bfd6c0d2b084418483a72536fbb6716a/users/97914dfbd4994558a3df3c3d6fb645f5/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:56.248708 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccba29-feda-453f-b1ac-b824b99e29d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfd6c0d2b084418483a72536fbb6716a, user_id=97914dfbd4994558a3df3c3d6fb645f5, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:56.251462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ccba29-feda-453f-b1ac-b824b99e29d9 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:56.268087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1432/2901] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:11:56 2022] PUT /identity/v3/projects/bfd6c0d2b084418483a72536fbb6716a/users/97914dfbd4994558a3df3c3d6fb645f5/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:11:56.273739 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547a691-2ad8-4add-9c05-a2967b7963dc None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:56.274024 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547a691-2ad8-4add-9c05-a2967b7963dc None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:56.274330 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8547a691-2ad8-4add-9c05-a2967b7963dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:56.294734 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-8547a691-2ad8-4add-9c05-a2967b7963dc None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:56.296072 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-8547a691-2ad8-4add-9c05-a2967b7963dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:56.360040 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1470/2902] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:56.370304 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccba29-feda-453f-b1ac-b824b99e29d9 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:56.372396 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e3ccba29-feda-453f-b1ac-b824b99e29d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:56.434392 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ccba29-feda-453f-b1ac-b824b99e29d9 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:56.438222 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443b9c9-0cb0-4db1-ba7b-6d55bb7a3d35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:56.438867 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443b9c9-0cb0-4db1-ba7b-6d55bb7a3d35 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:56.439147 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443b9c9-0cb0-4db1-ba7b-6d55bb7a3d35 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:56.439413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e443b9c9-0cb0-4db1-ba7b-6d55bb7a3d35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:56.448852 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443b9c9-0cb0-4db1-ba7b-6d55bb7a3d35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:56.450305 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e443b9c9-0cb0-4db1-ba7b-6d55bb7a3d35 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:56.460698 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1433/2903] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:57.485716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202fcc3-a282-46aa-a3f8-1f15b22397f6 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:57.486260 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202fcc3-a282-46aa-a3f8-1f15b22397f6 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:57.486404 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202fcc3-a282-46aa-a3f8-1f15b22397f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:57.527589 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-0202fcc3-a282-46aa-a3f8-1f15b22397f6 None None] MFA Rules not processed for user `97914dfbd4994558a3df3c3d6fb645f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:57.528820 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-0202fcc3-a282-46aa-a3f8-1f15b22397f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:57.579206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=100 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:57.617975 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1471/2904] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:57 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:57.628913 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443b9c9-0cb0-4db1-ba7b-6d55bb7a3d35 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:57.630462 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-e443b9c9-0cb0-4db1-ba7b-6d55bb7a3d35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:57.649620 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=2,keystone:SELECT=92,keystone:INSERT=6 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:11:57.692506 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e443b9c9-0cb0-4db1-ba7b-6d55bb7a3d35 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:57.696223 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc10708-4208-4caf-9e62-60e730b0abf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:57.696829 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10708-4208-4caf-9e62-60e730b0abf2 service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:57.697135 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10708-4208-4caf-9e62-60e730b0abf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:57.697441 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc10708-4208-4caf-9e62-60e730b0abf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:57.706616 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc10708-4208-4caf-9e62-60e730b0abf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:57.708110 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc10708-4208-4caf-9e62-60e730b0abf2 service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:57.717150 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1434/2905] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:11:59.114244 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2a75c-ca23-4baf-93e9-3ac150f76617 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:59.114808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2a75c-ca23-4baf-93e9-3ac150f76617 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:59.114808 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f2a75c-ca23-4baf-93e9-3ac150f76617 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:59.133887 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-29f2a75c-ca23-4baf-93e9-3ac150f76617 None None] MFA Rules not processed for user `97914dfbd4994558a3df3c3d6fb645f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:11:59.135206 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-29f2a75c-ca23-4baf-93e9-3ac150f76617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:59.142426 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1472/2906] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:11:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:11:59.825696 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc10708-4208-4caf-9e62-60e730b0abf2 service cinder] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:11:59.835522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc10708-4208-4caf-9e62-60e730b0abf2 service cinder] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:11:59.840024 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:11:59.840709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:11:59.841022 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:11:59.841286 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:11:59.843197 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:11:59.854478 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:11:59.856087 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:11:59.866399 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1435/2907] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:11:59 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:12:00.221295 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2a75c-ca23-4baf-93e9-3ac150f76617 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:12:00.223038 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-29f2a75c-ca23-4baf-93e9-3ac150f76617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:12:00.305914 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f2a75c-ca23-4baf-93e9-3ac150f76617 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:12:00.311190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cf8d9-f4dd-4067-82b0-9616cc535c84 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:12:00.311803 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cf8d9-f4dd-4067-82b0-9616cc535c84 service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:12:00.312082 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cf8d9-f4dd-4067-82b0-9616cc535c84 service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:12:00.312350 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96cf8d9-f4dd-4067-82b0-9616cc535c84 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:12:00.314147 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-d96cf8d9-f4dd-4067-82b0-9616cc535c84 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:12:00.325833 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cf8d9-f4dd-4067-82b0-9616cc535c84 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:12:00.327479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96cf8d9-f4dd-4067-82b0-9616cc535c84 service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:12:00.337670 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1473/2908] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:12:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:12:03.305989 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:12:03.307964 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:12:03.326120 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f242ec2-a368-4638-a2ad-fda8b82fcca9 service neutron] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:12:03.340263 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:12:03.343955 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:12:03.344653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:12:03.345232 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] PATH_INFO: `/v3/users/67fa873d16414749bc478a59de3ec5b4` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:12:03.347775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67fa873d16414749bc478a59de3ec5b4)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:12:03.349939 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:12:03.405364 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67fa873d16414749bc478a59de3ec5b4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.414295 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67fa873d16414749bc478a59de3ec5b4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.429379 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67fa873d16414749bc478a59de3ec5b4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.430117 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67fa873d16414749bc478a59de3ec5b4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.438315 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67fa873d16414749bc478a59de3ec5b4'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.444376 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1436/2909] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:12:03 2022] DELETE /identity/v3/users/67fa873d16414749bc478a59de3ec5b4 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:12:03.628876 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cf8d9-f4dd-4067-82b0-9616cc535c84 service placement] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:12:03.640453 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96cf8d9-f4dd-4067-82b0-9616cc535c84 service placement] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:12:03.710327 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:12:03.710989 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:12:03.711296 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:12:03.711585 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] PATH_INFO: `/v3/projects/99bbe5cbd59f4feda4da45f2b14bd09b` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:12:03.712735 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] RBAC: Authorizing `identity:delete_project(project_id=99bbe5cbd59f4feda4da45f2b14bd09b)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:12:03.714851 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:12:03.766145 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99bbe5cbd59f4feda4da45f2b14bd09b', 'request_id': 'req-77efe253-cc38-4cf1-a80d-b09e5274e78d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.772112 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99bbe5cbd59f4feda4da45f2b14bd09b', 'request_id': 'req-77efe253-cc38-4cf1-a80d-b09e5274e78d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.778163 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99bbe5cbd59f4feda4da45f2b14bd09b', 'request_id': 'req-77efe253-cc38-4cf1-a80d-b09e5274e78d'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.791718 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:12:03.796897 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] The token cache is being invalidate because project 99bbe5cbd59f4feda4da45f2b14bd09b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:12:03.797414 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.798721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.800798 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1474/2910] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:12:03 2022] DELETE /identity/v3/projects/99bbe5cbd59f4feda4da45f2b14bd09b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:12:03.804941 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:12:03.806860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:12:03.820860 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc193c-b403-4077-a6cf-b1e2e72a04d3 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:12:03.884576 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:12:03.885319 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:12:03.885598 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:12:03.885870 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] PATH_INFO: `/v3/users/f5d33a38add34abaa240270dc11af380` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:12:03.887075 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5d33a38add34abaa240270dc11af380)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:12:03.889014 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:12:03.911500 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:12:03.943242 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5d33a38add34abaa240270dc11af380'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.951868 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5d33a38add34abaa240270dc11af380'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.963717 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5d33a38add34abaa240270dc11af380'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.964346 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5d33a38add34abaa240270dc11af380'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.971067 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5d33a38add34abaa240270dc11af380'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:03.976828 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1437/2911] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:12:03 2022] DELETE /identity/v3/users/f5d33a38add34abaa240270dc11af380 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:12:04.171712 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:12:04.173902 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:12:04.188481 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efe253-cc38-4cf1-a80d-b09e5274e78d admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:12:04.247771 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:12:04.248412 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:12:04.248688 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:12:04.248941 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] PATH_INFO: `/v3/projects/6eae938afce74856b6dfb2dede2b59cc` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:12:04.250105 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eae938afce74856b6dfb2dede2b59cc)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:12:04.252031 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:12:04.304042 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eae938afce74856b6dfb2dede2b59cc', 'request_id': 'req-24baefcd-334a-43a2-9130-f85cb0f6374c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:04.310597 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eae938afce74856b6dfb2dede2b59cc', 'request_id': 'req-24baefcd-334a-43a2-9130-f85cb0f6374c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:04.317545 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eae938afce74856b6dfb2dede2b59cc', 'request_id': 'req-24baefcd-334a-43a2-9130-f85cb0f6374c'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:04.333349 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] The token cache is being invalidate because project 6eae938afce74856b6dfb2dede2b59cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81500) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:12:04.333769 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:04.334449 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:12:04.336009 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1475/2912] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:12:04 2022] DELETE /identity/v3/projects/6eae938afce74856b6dfb2dede2b59cc => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:12:09.186909 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:12:09.188499 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:12:09.265754 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c616472-3a4c-4f8b-bda8-fbe2f68173f7 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:12:09.332390 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da599513-201f-4249-86e9-6e872c42170b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:12:09.333017 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da599513-201f-4249-86e9-6e872c42170b service nova] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:12:09.333299 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da599513-201f-4249-86e9-6e872c42170b service nova] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:12:09.333571 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da599513-201f-4249-86e9-6e872c42170b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:12:09.335472 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-da599513-201f-4249-86e9-6e872c42170b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:12:09.348314 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da599513-201f-4249-86e9-6e872c42170b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:12:09.349928 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da599513-201f-4249-86e9-6e872c42170b service nova] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:12:09.360802 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1438/2913] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:12:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:12:14.330380 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=52,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:12:19.356160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:DELETE=8,keystone:SELECT=82,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:12:53.375428 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:12:53.377054 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:12:53.394048 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baefcd-334a-43a2-9130-f85cb0f6374c admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:12:53.400681 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '87f3a9ffb88f4faa85edc3dabce0c8f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'service', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:12:53.401279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:12:53.401552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:12:53.401816 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:12:53.404556 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:12:53.417750 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:12:53.419379 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:12:53.430952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1476/2914] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:12:53 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:13:02.530709 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8631a9a-6b57-4d22-98c0-10f0bb224129 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:02.531282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8631a9a-6b57-4d22-98c0-10f0bb224129 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:02.531282 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8631a9a-6b57-4d22-98c0-10f0bb224129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:02.551775 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d8631a9a-6b57-4d22-98c0-10f0bb224129 None None] MFA Rules not processed for user `97914dfbd4994558a3df3c3d6fb645f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:13:02.552980 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d8631a9a-6b57-4d22-98c0-10f0bb224129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:13:02.619070 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1439/2915] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:13:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:13:03.426012 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=6 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:13:12.611771 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=25 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:13:57.728733 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:13:57.730651 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:13:57.808721 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2a6e65-dfb7-4905-a760-4039ba5f5fda service nova] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:13:57.813702 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:13:57.814389 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:57.814647 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:57.814893 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] PATH_INFO: `/v3/users/97914dfbd4994558a3df3c3d6fb645f5` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:57.816029 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97914dfbd4994558a3df3c3d6fb645f5)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:13:57.817965 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:13:57.874190 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97914dfbd4994558a3df3c3d6fb645f5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:57.874844 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97914dfbd4994558a3df3c3d6fb645f5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:57.885824 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97914dfbd4994558a3df3c3d6fb645f5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:57.903058 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97914dfbd4994558a3df3c3d6fb645f5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:57.903782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-f2486438-a407-4e1c-b28c-fe249ed88c74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97914dfbd4994558a3df3c3d6fb645f5'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:57.917595 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1477/2916] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:13:57 2022] DELETE /identity/v3/users/97914dfbd4994558a3df3c3d6fb645f5 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:13:58.129008 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8631a9a-6b57-4d22-98c0-10f0bb224129 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:13:58.143047 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8631a9a-6b57-4d22-98c0-10f0bb224129 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:13:58.202522 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:13:58.203160 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.203427 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.203706 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] PATH_INFO: `/v3/projects/bfd6c0d2b084418483a72536fbb6716a` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.204840 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfd6c0d2b084418483a72536fbb6716a)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:13:58.206779 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:13:58.257233 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfd6c0d2b084418483a72536fbb6716a', 'request_id': 'req-bfe2d732-582e-413c-9540-1571b9dfb83a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:58.263988 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfd6c0d2b084418483a72536fbb6716a', 'request_id': 'req-bfe2d732-582e-413c-9540-1571b9dfb83a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:58.270523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfd6c0d2b084418483a72536fbb6716a', 'request_id': 'req-bfe2d732-582e-413c-9540-1571b9dfb83a'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:58.285895 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] The token cache is being invalidate because project bfd6c0d2b084418483a72536fbb6716a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:13:58.286221 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:58.286865 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-bfe2d732-582e-413c-9540-1571b9dfb83a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:13:58.288434 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1440/2917] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:13:58 2022] DELETE /identity/v3/projects/bfd6c0d2b084418483a72536fbb6716a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:13:58.293605 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827a50c-40a0-43d1-9457-ae3f67fbd1c0 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.293900 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827a50c-40a0-43d1-9457-ae3f67fbd1c0 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.294209 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827a50c-40a0-43d1-9457-ae3f67fbd1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.314565 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-3827a50c-40a0-43d1-9457-ae3f67fbd1c0 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:13:58.315782 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-3827a50c-40a0-43d1-9457-ae3f67fbd1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:13:58.389788 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1478/2918] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 13 07:13:58.398784 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57b92-5661-48b9-8f3f-8a71c174e3a7 None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.399061 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57b92-5661-48b9-8f3f-8a71c174e3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.399409 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff57b92-5661-48b9-8f3f-8a71c174e3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.421412 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-2ff57b92-5661-48b9-8f3f-8a71c174e3a7 None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:13:58.422668 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-2ff57b92-5661-48b9-8f3f-8a71c174e3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:13:58.432523 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1441/2919] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:13:58.437920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827a50c-40a0-43d1-9457-ae3f67fbd1c0 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:13:58.449695 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827a50c-40a0-43d1-9457-ae3f67fbd1c0 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:13:58.454972 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e74e91-147d-41ad-bc5a-e29b110ac437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:13:58.455581 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e74e91-147d-41ad-bc5a-e29b110ac437 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.455862 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e74e91-147d-41ad-bc5a-e29b110ac437 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.456133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e74e91-147d-41ad-bc5a-e29b110ac437 admin admin] PATH_INFO: `/v3/domains` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.456780 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e74e91-147d-41ad-bc5a-e29b110ac437 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81500) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 13 07:13:58.457083 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e74e91-147d-41ad-bc5a-e29b110ac437 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:13:58.458716 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e74e91-147d-41ad-bc5a-e29b110ac437 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:13:58.471725 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1479/2920] 10.222.0.25 () {58 vars in 1153 bytes} [Tue Sep 13 07:13:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 13 07:13:58.477653 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57b92-5661-48b9-8f3f-8a71c174e3a7 None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:13:58.489329 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff57b92-5661-48b9-8f3f-8a71c174e3a7 None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:13:58.498327 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a648e-6f05-4bec-a7cb-07815f004d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:13:58.499208 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a648e-6f05-4bec-a7cb-07815f004d7b admin admin] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.499536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a648e-6f05-4bec-a7cb-07815f004d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.499894 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a648e-6f05-4bec-a7cb-07815f004d7b admin admin] PATH_INFO: `/v3/projects` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.500845 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a648e-6f05-4bec-a7cb-07815f004d7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:13:58.503930 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a648e-6f05-4bec-a7cb-07815f004d7b admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:13:58.545963 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1442/2921] 10.222.0.25 () {60 vars in 1153 bytes} [Tue Sep 13 07:13:58 2022] POST /identity/v3/projects => generated 391 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:13:58.551719 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e74e91-147d-41ad-bc5a-e29b110ac437 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:13:58.562442 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e74e91-147d-41ad-bc5a-e29b110ac437 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:13:58.566275 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:13:58.622514 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3a45a-fcc4-4cef-80d8-dfbe10e00795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:13:58.623125 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3a45a-fcc4-4cef-80d8-dfbe10e00795 admin admin] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.623400 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3a45a-fcc4-4cef-80d8-dfbe10e00795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.623673 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e3a45a-fcc4-4cef-80d8-dfbe10e00795 admin admin] PATH_INFO: `/v3/users` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.624318 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3a45a-fcc4-4cef-80d8-dfbe10e00795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:13:58.626256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e3a45a-fcc4-4cef-80d8-dfbe10e00795 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:13:58.650133 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1480/2922] 10.222.0.25 () {60 vars in 1144 bytes} [Tue Sep 13 07:13:58 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 13 07:13:58.656873 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a648e-6f05-4bec-a7cb-07815f004d7b admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:13:58.669477 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a648e-6f05-4bec-a7cb-07815f004d7b admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:13:58.674536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7d2d2-b17e-482b-8f39-138175df7753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:13:58.675143 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7d2d2-b17e-482b-8f39-138175df7753 admin admin] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.675443 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7d2d2-b17e-482b-8f39-138175df7753 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.675711 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a7d2d2-b17e-482b-8f39-138175df7753 admin admin] PATH_INFO: `/v3/roles` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.676348 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7d2d2-b17e-482b-8f39-138175df7753 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:13:58.678071 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a7d2d2-b17e-482b-8f39-138175df7753 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:13:58.689665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1443/2923] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:13:58.693773 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3a45a-fcc4-4cef-80d8-dfbe10e00795 admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:13:58.705283 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e3a45a-fcc4-4cef-80d8-dfbe10e00795 admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:13:58.711246 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d4d8d6-9c24-44be-b933-9a39afafc833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:13:58.711922 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4d8d6-9c24-44be-b933-9a39afafc833 admin admin] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.712196 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4d8d6-9c24-44be-b933-9a39afafc833 admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.712480 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d4d8d6-9c24-44be-b933-9a39afafc833 admin admin] PATH_INFO: `/v3/roles` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.713256 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4d8d6-9c24-44be-b933-9a39afafc833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:13:58.714944 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d4d8d6-9c24-44be-b933-9a39afafc833 admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:13:58.727231 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1481/2924] 10.222.0.25 () {58 vars in 1122 bytes} [Tue Sep 13 07:13:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 13 07:13:58.731942 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7d2d2-b17e-482b-8f39-138175df7753 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:13:58.742665 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a7d2d2-b17e-482b-8f39-138175df7753 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:13:58.748973 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb7c1-b392-4ea8-94e4-a5db8cae629a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:13:58.749630 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb7c1-b392-4ea8-94e4-a5db8cae629a admin admin] REQUEST_METHOD: `PUT` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.749904 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb7c1-b392-4ea8-94e4-a5db8cae629a admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.750210 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318eb7c1-b392-4ea8-94e4-a5db8cae629a admin admin] PATH_INFO: `/v3/projects/cc1d137761b64e19b15b94945095a99e/users/23c6d2321478485e986d4d3390cce7b2/roles/2d59fcad66104464bbc848ef605f743e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.765677 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eb7c1-b392-4ea8-94e4-a5db8cae629a admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1d137761b64e19b15b94945095a99e, user_id=23c6d2321478485e986d4d3390cce7b2, role_id=2d59fcad66104464bbc848ef605f743e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:13:58.768370 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318eb7c1-b392-4ea8-94e4-a5db8cae629a admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:13:58.786016 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1444/2925] 10.222.0.25 () {60 vars in 1483 bytes} [Tue Sep 13 07:13:58 2022] PUT /identity/v3/projects/cc1d137761b64e19b15b94945095a99e/users/23c6d2321478485e986d4d3390cce7b2/roles/2d59fcad66104464bbc848ef605f743e => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:13:58.796445 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f353d9e-1502-4962-9ebb-3e2223616ce9 None None] REQUEST_METHOD: `POST` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.796983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f353d9e-1502-4962-9ebb-3e2223616ce9 None None] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.796983 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f353d9e-1502-4962-9ebb-3e2223616ce9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.825726 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.auth.core [None req-4f353d9e-1502-4962-9ebb-3e2223616ce9 None None] MFA Rules not processed for user `23c6d2321478485e986d4d3390cce7b2`. Rule list: `[]` (Enabled: `True`). {{(pid=81500) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:13:58.826952 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-4f353d9e-1502-4962-9ebb-3e2223616ce9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:13:58.891723 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1482/2926] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:13:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:13:58.902781 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb7c1-b392-4ea8-94e4-a5db8cae629a admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:13:58.904595 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-318eb7c1-b392-4ea8-94e4-a5db8cae629a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:13:58.936323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:13:58.964293 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318eb7c1-b392-4ea8-94e4-a5db8cae629a admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:13:58.968525 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51fa97a-e4da-4502-a01b-ea3e43f60a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '88520ce6493a44f39b19c67592f03632', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:13:58.969134 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fa97a-e4da-4502-a01b-ea3e43f60a3c service cinder] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:13:58.969413 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fa97a-e4da-4502-a01b-ea3e43f60a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:13:58.969684 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51fa97a-e4da-4502-a01b-ea3e43f60a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:13:58.978045 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fa97a-e4da-4502-a01b-ea3e43f60a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:13:58.979536 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51fa97a-e4da-4502-a01b-ea3e43f60a3c service cinder] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:13:58.988271 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1445/2927] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:13:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:14:08.888857 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=63,keystone:INSERT=3 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:14:08.984422 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=14 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:14:15.452714 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f353d9e-1502-4962-9ebb-3e2223616ce9 None None] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:14:15.518694 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f353d9e-1502-4962-9ebb-3e2223616ce9 None None] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:14:15.523572 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:14:15.524202 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:14:15.524487 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:14:15.524762 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] PATH_INFO: `/v3/users/23c6d2321478485e986d4d3390cce7b2` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:14:15.525920 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] RBAC: Authorizing `identity:delete_user(user_id=23c6d2321478485e986d4d3390cce7b2)` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:14:15.527882 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:14:15.586477 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23c6d2321478485e986d4d3390cce7b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:15.587360 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23c6d2321478485e986d4d3390cce7b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:15.598398 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23c6d2321478485e986d4d3390cce7b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:15.611745 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23c6d2321478485e986d4d3390cce7b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:15.612907 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.notifications [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23c6d2321478485e986d4d3390cce7b2'} {{(pid=81500) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:15.625981 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1483/2928] 10.222.0.25 () {58 vars in 1224 bytes} [Tue Sep 13 07:14:15 2022] DELETE /identity/v3/users/23c6d2321478485e986d4d3390cce7b2 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:14:15.630701 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931b518-ca6f-4fa2-a3c6-65fd7c7aef2a None None] REQUEST_METHOD: `POST` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:14:15.631176 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931b518-ca6f-4fa2-a3c6-65fd7c7aef2a None None] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:14:15.631290 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d931b518-ca6f-4fa2-a3c6-65fd7c7aef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:14:15.650874 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.auth.core [None req-d931b518-ca6f-4fa2-a3c6-65fd7c7aef2a None None] MFA Rules not processed for user `d3bdb62de62349b6b321bb705e25f4be`. Rule list: `[]` (Enabled: `True`). {{(pid=81499) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 13 07:14:15.652089 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d931b518-ca6f-4fa2-a3c6-65fd7c7aef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:14:15.719313 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1446/2929] 10.222.0.25 () {58 vars in 958 bytes} [Tue Sep 13 07:14:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 13 07:14:15.729225 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:14:15.730839 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:14:15.791749 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add2a39a-ba82-4720-9f51-c948e5a9a70a admin admin] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:14:15.795877 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d05ce-3e42-4d10-b409-328c1e7fee86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f94aa8af227a42a08d8a0e4b7f02b85a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:14:15.796479 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d05ce-3e42-4d10-b409-328c1e7fee86 service neutron] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:14:15.796752 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d05ce-3e42-4d10-b409-328c1e7fee86 service neutron] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:14:15.797005 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767d05ce-3e42-4d10-b409-328c1e7fee86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:14:15.806690 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d05ce-3e42-4d10-b409-328c1e7fee86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:14:15.808937 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-767d05ce-3e42-4d10-b409-328c1e7fee86 service neutron] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:14:15.823333 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1484/2930] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:14:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:14:16.099011 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931b518-ca6f-4fa2-a3c6-65fd7c7aef2a None None] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:14:16.100664 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-d931b518-ca6f-4fa2-a3c6-65fd7c7aef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:14:16.118054 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d931b518-ca6f-4fa2-a3c6-65fd7c7aef2a None None] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:14:16.123398 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd3bdb62de62349b6b321bb705e25f4be', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2becfff2f1104b709c0a7babcac266dd', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:14:16.123875 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:14:16.124137 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:14:16.124446 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] PATH_INFO: `/v3/projects/cc1d137761b64e19b15b94945095a99e` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:14:16.125741 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1d137761b64e19b15b94945095a99e)` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:14:16.127555 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:14:16.186753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1d137761b64e19b15b94945095a99e', 'request_id': 'req-36caa752-72e7-49e2-ba7d-780f1b485430'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:16.193504 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1d137761b64e19b15b94945095a99e', 'request_id': 'req-36caa752-72e7-49e2-ba7d-780f1b485430'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:16.199296 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1d137761b64e19b15b94945095a99e', 'request_id': 'req-36caa752-72e7-49e2-ba7d-780f1b485430'} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:16.214470 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] The token cache is being invalidate because project cc1d137761b64e19b15b94945095a99e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81499) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 13 07:14:16.214753 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:16.215380 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.notifications [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81499) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 13 07:14:16.217200 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1447/2931] 10.222.0.25 () {58 vars in 1233 bytes} [Tue Sep 13 07:14:16 2022] DELETE /identity/v3/projects/cc1d137761b64e19b15b94945095a99e => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 13 07:14:21.737855 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d05ce-3e42-4d10-b409-328c1e7fee86 service neutron] Authenticating user token {{(pid=81500) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:14:21.739548 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-767d05ce-3e42-4d10-b409-328c1e7fee86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:14:21.771488 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:14:21.821109 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767d05ce-3e42-4d10-b409-328c1e7fee86 service neutron] Validating token access rules against request {{(pid=81500) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:14:21.893717 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67af49d2-1994-4c83-a962-5d4e4ce6a07e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81500) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:14:21.894279 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af49d2-1994-4c83-a962-5d4e4ce6a07e service placement] REQUEST_METHOD: `GET` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:14:21.894552 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af49d2-1994-4c83-a962-5d4e4ce6a07e service placement] SCRIPT_NAME: `/identity` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:14:21.894835 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67af49d2-1994-4c83-a962-5d4e4ce6a07e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81500) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:14:21.896776 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.fernet_utils [None req-67af49d2-1994-4c83-a962-5d4e4ce6a07e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81500) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:14:21.908265 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af49d2-1994-4c83-a962-5d4e4ce6a07e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:14:21.909846 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67af49d2-1994-4c83-a962-5d4e4ce6a07e service placement] RBAC: Authorization granted {{(pid=81500) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:14:21.920471 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: [pid: 81500|app: 0|req: 1485/2932] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:14:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 13 07:14:26.211476 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG dbcounter [-] [81499] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81499) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:14:31.916276 dv-cinder-856941-4-iscsi devstack@keystone.service[81500]: DEBUG dbcounter [-] [81500] Writing DB stats keystone:SELECT=48 {{(pid=81500) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 13 07:19:27.729800 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] Authenticating user token {{(pid=81499) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 13 07:19:27.732179 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.fernet_utils [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81499) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 13 07:19:27.827171 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36caa752-72e7-49e2-ba7d-780f1b485430 admin admin] Validating token access rules against request {{(pid=81499) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 13 07:19:27.833039 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e8a9c-b0be-49ba-96b8-2759ba169fac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e8202ad6007246078c4d25365dc5a0f5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd05778d5266247d29f296ce5c9792b7e', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81499) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 13 07:19:27.833652 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8a9c-b0be-49ba-96b8-2759ba169fac service placement] REQUEST_METHOD: `GET` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 13 07:19:27.833927 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8a9c-b0be-49ba-96b8-2759ba169fac service placement] SCRIPT_NAME: `/identity` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 13 07:19:27.834231 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e8a9c-b0be-49ba-96b8-2759ba169fac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81499) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 13 07:19:27.843739 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e8a9c-b0be-49ba-96b8-2759ba169fac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 13 07:19:27.845323 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e8a9c-b0be-49ba-96b8-2759ba169fac service placement] RBAC: Authorization granted {{(pid=81499) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 13 07:19:27.856107 dv-cinder-856941-4-iscsi devstack@keystone.service[81499]: [pid: 81499|app: 0|req: 1448/2933] 10.222.0.25 () {60 vars in 1365 bytes} [Tue Sep 13 07:19:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)